OshonSoft AI Language Model Lab
Bracket Logic
PRODUCT HOMEPAGE - GUIDED EXAMPLES AND SCREENSHOTS - USER MANUAL
Train a small model to check nested brackets and show each step.
Give the model a string made up of () and []. Follow the open brackets as it reads the input from left to right. The reply gives a verdict. It also gives the first error position if a bracket closes incorrectly, or the closing brackets still needed if the sequence is incomplete.
The final MLP has 26,910 parameters and learns from 4,000 training conversations. The selected model answered 1,200/1,200 frozen Final evaluation questions in full, including the end of turn.
BRACKET LOGIC - THREE-DIGIT ADDITION - TRANSLATION PATTERNS
On this page
1. Try the trained model
2. Read the bracket trace
3. Compare saved replies during training
4. See what changed with more practice
5. Repeat the training
1. Try the trained model
The download contains the primary and confirmation projects, their initial and trained models, and complete saved replies. Both runs reproduced 1,200/1,200 exact Final evaluation replies with the current Lab training and generation code.
DOWNLOAD PROJECTS AND MODELS - DOWNLOAD THE LAB
The trained model is supplied as models/selected.osai in its project. Load it through Persistence, using Choose load file and Load checkpoint. In Conversation, choose Greedy, set the reply limit shown below and clear the conversation before each independent question.
Set Reply tokens to 512. Leave Bracket trace on. Enter a bracket string and click Send / continue reply. The final reply should end with [EOT].
Click either screenshot to open the original at full size.
![Conversation tab showing ([]) and its generated trace through EOT](images/bracket-valid.png)
![Conversation tab showing ()][ and its generated trace through EOT](images/bracket-invalid.png)
WATCH THE MODEL GENERATE THE TRACE (MP4)
Try the two original ten-character examples. Both are new to this model. These are the last lines of the saved native replies.
USER ([])[]([]] result: invalid at 10, expected ). USER ([])[]([]) result: valid.
2. Read the bracket trace
The input contains one to ten bracket characters. The model reports the length and then advances one bracket at a time. Positions start at 1.
The left field shows the open brackets, most recent first. The right field shows the remaining input. Dashes fill empty cells. The count line gives the model a short counting sequence to align its copy of the input.
A wrong closer gives an invalid result with its position and expected closer. If there was nothing open, the result says expected none. An incomplete result lists the closers still needed, in the order they should be added.
Saved reply excerpt for a new input
USER [[()[[([(] … 8 [: [([[[[---- | (]-------- 9 (: ([([[[[--- | ]--------- 10 ]: error ( result: invalid at 10, expected ).
3. Compare saved replies during training
Choose a Training checks question and a checkpoint. Compare the saved model reply with the reference answer. The Lab generated each reply in a fresh conversation from the user input. Both examples are still incorrect at the 200,000-update checkpoint and become correct by epoch 1. Both answers use the same Bracket trace spacing as the application.
Saved model reply
length: 8 count: 8 9 10 11 12 start: - | [[[[[(([-- 1 [: [--------- | [[[[(([--- 2 [: [[-------- | [[[(([---- 3 [: [[[------- | [[(([----- 4 [: [[[[------ | [(([------ 5 [: [[[[[----- | (([------- 6 (: ([[[[[---- | ([-------- 7 (: (([[[[[--- | [--------- 8 [: [(([[[---- | ---------- result: incomplete, expected ]))]]].
Reference answer
length: 8 count: 8 9 10 11 12 start: - | [[[[[(([-- 1 [: [--------- | [[[[(([--- 2 [: [[-------- | [[[(([---- 3 [: [[[------- | [[(([----- 4 [: [[[[------ | [(([------ 5 [: [[[[[----- | (([------- 6 (: ([[[[[---- | ([-------- 7 (: (([[[[[--- | [--------- 8 [: [(([[[[[-- | ---------- result: incomplete, expected ]))]]]]].
4. See what changed with more practice
The same random sampling rules were used for TRAIN, Training checks and Final evaluation within each recipe. TRAIN contained 4,000 sequences. Training checks contained 600 new sequences for choosing the checkpoint. Final evaluation contained 1,200 new sequences and was opened after that choice. A sequence and the same sequence with round and square brackets swapped share one split identity.
A whole reply was counted as correct when every token matched the reference and the model generated the end-of-turn token. For errors, we also checked whether the prefix up to the first error had appeared in TRAIN.
On the original 4,000 examples, the 26,910-parameter MLP reached 1,199 of 1,200 on two independent tests. In one comparison the same deep closing step failed after four and six epochs. The original training material had two examples of a legal close at position 10 with eight openings left.
The preceding deep-close recipe reached 100% with seed 12345. Its confirmation missed two Final evaluation traces with ten simultaneous openings. TRAIN contained only six such sequences. The next experiment increased their frequency and used new questions.
The final sampler draws all three folds with the same rules. It makes deep opens and closes more frequent. TRAIN contains 225 sequences with ten simultaneous openings and 64 legal closes at position 10 with eight openings left.
| TRAIN material | Model | Epochs | Whole Training checks | Whole Final evaluation |
|---|---|---|---|---|
| Random 2,000 | MLP · 100,820 | 4 | 596/600 | 1,192/1,200 |
| Random 2,000 | MLP · 26,910 | 4 | 596/600 | 1,191/1,200 |
| Random 4,000 | MLP · 100,820 | 2 | 600/600 | 1,198/1,200 |
| Random 4,000 | MLP · 26,910 | 2 | 600/600 | 1,198/1,200 |
| Random 4,000 · shared comparison | MLP · 26,910 | 4 | 600/600 | 1,199/1,200 |
| Random 4,000 · shared comparison | MLP · 26,910 | 6 | 600/600 | 1,199/1,200 |
| Random 4,000 · shared comparison | Transformer · 25,720 | 1 | 416/600 | 791/1,200 |
| Random 4,000 · deep opens and closes | MLP · 26,910 | 6 | 600/600 | 1,200/1,200 |
Models on the same named Final evaluation suite used the same questions. Each material recipe had its own frozen suite. The Transformer row shows the checkpoint selected on Training checks after one epoch of a two-epoch run.
Final recipe, seed 12345
| Checkpoint | Updates | Whole Training checks | End of turn |
|---|---|---|---|
| Before training | 0 | 0/600 | 287/600 |
| 200,000 updates | 200,000 | 484/600 | 600/600 |
| Epoch 1 | 784,721 | 600/600 | 600/600 |
| Epoch 2 | 1,569,442 | 600/600 | 600/600 |
| Epoch 3 | 2,354,163 | 600/600 | 600/600 |
| Epoch 4 | 3,138,884 | 600/600 | 600/600 |
| Epoch 5 | 3,923,605 | 600/600 | 600/600 |
| Epoch 6 | 4,708,326 | 600/600 | 600/600 |
Independent confirmation
Seed 23456 started with fresh weights and the same settings. It reached 600/600 on Training checks, generated 1,200/1,200 whole Final evaluation replies, and ended 1,200/1,200 Final evaluation replies correctly.
Known TRAIN replies were exact in 4,000/4,000 cases.
| Primary Final evaluation detail | Count |
|---|---|
| Complete replies | 1,200/1,200 |
| Even lengths | 733/733 |
| Odd lengths | 467/467 |
| Correct end of turn | 1,200/1,200 |
| Detected wrong or unexpected close | 930/930 |
| Exact first-error position | 930/930 |
| Whole replies on new error prefixes | 217/217 |
| Equal-symbol-count contrast pairs | 2/2 |
| Whole replies on known TRAIN examples | 4,000/4,000 |
The results cover the frozen questions and bracket strings up to ten characters.
5. Repeat the training
On an AMD Ryzen 7 7735HS laptop, the repeated primary run took about 2 h 46 min. These times include training and all configured checks while several projects shared the CPU. Time spent paused is excluded. Other hardware and workloads will give different timings.
Extract bracket_logic into your writable Documents/OshonSoft AI Language Model Lab folder. It contains the original twenty training files, Training checks, separate Final evaluation questions and a current project definition.
Use MLP V40 / C28 / D12 / H70, seed 12345, Custom embedding range with half-range 2.0, Words and punctuation, Preserve line breaks, Global epochs, MODEL body/EOT targets and SGD 0.006.
- Open the installed OshonSoft AI Language Model Lab. On Project, click Choose project... and select the folder containing
experiment.json. The saved settings fill the controls; this does not start training. - Review the dimensions, tokenizer, initialization, learning rate and target. Click Prepare and initialize model. Use the prepared target count to read the update budget.
- Start a fresh run. The Lab saves checkpoints and full Prompt/Gold/Raw replies under
runs/run-000001/. Use Open run folder or Read replies... to inspect them. - Use Pause + save before closing. Later choose the saved
run.json, click Open / recover run, then Resume. Resume keeps the original target.
The project uses 4,708,326 updates, or six epochs of 784,721 targets. Its extra checkpoints are 200000,784721,1569442,2354163,3138884,3923605; zero and the endpoint are included. Reply limit: 512. The supplied model is the sixth-epoch checkpoint; the independent confirmation uses seed 23456 with the same settings.
Use questions/ for Training checks. Select the checkpoint under the study's stated rule, then use Use checkpoint... and Final evaluation for questions/final/. Reusing these already evaluated question sets is a reproduction check; it is not a new unseen test.
The archive also includes bracket_logic_confirmation with seed 23456. Each project supplies models/initial.osai and models/selected.osai. Complete saved replies are grouped by update in answers/, with the final replies in answers/final/. A fresh run creates its own runs/ folder.
DOWNLOAD PROJECTS AND MODELS - PRODUCT HOMEPAGE - USER MANUAL