|
| 1 | +# circle2circle (circle to circle) |
| 2 | + |
| 3 | +`circle2circle` is a tool for transforming Circle models. |
| 4 | + |
| 5 | +It includes various filter command (= pass) to perform specific modifications. |
| 6 | + |
| 7 | +<br> |
| 8 | + |
| 9 | +## How to Use |
| 10 | + |
| 11 | +Imagine Unix filter usage like `cat hello.txt | sort | uniq`. |
| 12 | + |
| 13 | +All circle2circle command scripts read a Circle model from **standard input** and write the transformed model to **standard output**. |
| 14 | + |
| 15 | +An example: |
| 16 | + |
| 17 | +```bash |
| 18 | +./rename.io.remove_namespace.py < in.circle > out.circle |
| 19 | +``` |
| 20 | + |
| 21 | +Filters example: |
| 22 | + |
| 23 | +```bash |
| 24 | +./rename.io.remove_namespace.py < in.circle | ./rename.io.remove_prefix.py past_key_values_ > out.circle |
| 25 | +``` |
| 26 | + |
| 27 | +<br> |
| 28 | + |
| 29 | +## Filter List |
| 30 | + |
| 31 | +### `remove.io.py` |
| 32 | + |
| 33 | +Removes input or output tensors from a Circle model, keeping only the tensors at the specified indices. |
| 34 | + |
| 35 | +#### Arguments |
| 36 | + |
| 37 | +* `io_type` (required): Specifies whether to process `input` or `output` tensors. |
| 38 | +* `--keep_by_name` (required): A string defining the names of the tensors to keep. It supports comma‑separated tensor names (e.g., "input1,input2"). |
| 39 | + |
| 40 | +## |
| 41 | + |
| 42 | +### `rename.io.remove_namespace.py` |
| 43 | + |
| 44 | +Removes namespaces from the names of input and output tensors. A namespace is identified as the part of the tensor name before a double colon (`::`). For example, a tensor named `module::input_tensor` would be renamed to `input_tensor`. |
| 45 | + |
| 46 | +## |
| 47 | + |
| 48 | +### `rename.io.remove_prefix.py` |
| 49 | + |
| 50 | +Removes a user-specified prefix from the names of all tensors in the model. |
| 51 | + |
| 52 | +#### Arguments |
| 53 | + |
| 54 | +* `prefix` (required): The string prefix to remove from tensor names. |
| 55 | + |
| 56 | + |
| 57 | +## |
| 58 | + |
| 59 | +### `reshape.fc_weight.py` |
| 60 | + |
| 61 | +Reshapes the weight tensors of `FULLY_CONNECTED` operators from an effectively 2D shape (e.g., `[1, 1, D_out, D_in]`) to a strict 2D shape (`[D_out, D_in]`). This is useful for optimizing or standardizing the model structure. If a weight tensor is used by multiple operators, a new tensor is created for the specific operator to prevent conflicts. |
| 62 | + |
| 63 | +## |
| 64 | + |
| 65 | +### `transpose.io.kcache.py` |
| 66 | + |
| 67 | +Finds input tensors matching the pattern `*key_cache_\d+` (e.g., `past_key_values_key_cache_0`) and transposes their second and third dimensions if they are 4D. For example, a shape `[d0, d1, d2, d3]` will become `[d0, d2, d1, d3]`. |
| 68 | + |
| 69 | +## |
| 70 | + |
| 71 | +### `fuse.bmm_lhs_const.py` |
| 72 | + |
| 73 | +Fuses `BATCH_MATMUL` + `TRANSPOSE` to `FULLY_CONNECTED` when LHS is constant. |
| 74 | + |
| 75 | +#### Transformation Diagram |
| 76 | + |
| 77 | +``` |
| 78 | +BEFORE: |
| 79 | +
|
| 80 | +LHS(constant):[B,M,K] \ |
| 81 | + BatchMatMul(LHS,RHS):[B,M,N] -> TRANSPOSE:[B,N,M] -> OUTPUT |
| 82 | +RHS:[B,K,N] / |
| 83 | +
|
| 84 | +AFTER: |
| 85 | +
|
| 86 | +RHS:[B,K,N] \ |
| 87 | + FullyConnected(RHS,LHS):[B,N,M] -> OUTPUT |
| 88 | +LHS(constant):[B,M,K] / ~~ ~~ |
| 89 | + input weights |
| 90 | +
|
| 91 | +Condition: |
| 92 | +- B = 1 and K = 1 |
| 93 | +
|
| 94 | +Key Relationship: |
| 95 | +- BatchMatMul's LHS (constant) becomes FullyConnected's weights |
| 96 | +- BatchMatMul's RHS becomes FullyConnected's input |
| 97 | +``` |
| 98 | + |
| 99 | +## |
| 100 | + |
| 101 | +### `remove.unused_tensors.py` |
| 102 | + |
| 103 | +Identifies and removes unused tensors from all subgraphs within a Circle model. A tensor is considered "unused" if it is not an input to any operator and not an output of its containing subgraph. This helps in cleaning up the model and potentially reducing its size. The script can either list unused tensors or modify the model to remove them. |
| 104 | + |
| 105 | +## |
| 106 | + |
| 107 | +### `gen_circle.*.py` |
| 108 | + |
| 109 | + |
| 110 | +These scripts generate test Circle models with specific operator patterns for development and testing purposes. Each script follows the naming convention `gen_circle.<operation>.py` and automatically generates an output file with the name `<operation>.circle` when executed. |
| 111 | + |
| 112 | +#### `gen_circle.add.py` |
| 113 | + |
| 114 | +Generates a simple Circle model with one `ADD` operator for testing basic functionality. |
| 115 | + |
| 116 | +#### `gen_circle.bmm_lhs_const.fc.py` |
| 117 | + |
| 118 | +Generates a test Circle model with `BATCH_MATMUL` and `TRANSPOSE` operations where the LHS is constant. This model is designed to test the fusion pattern used in `fuse.bmm_lhs_const.py`. |
0 commit comments