Checking errors programatically.

Are you familiar with the concept of unit testing? https://en.m.wikipedia.org/wiki/Unit_testing

No, it is about collecting error messages to generate training data for a model.

Correct me if I'm wrong, but this sounds a bit like you're coding using A.I., getting errors, and then wanting A.I. to fix the list of errors...?

a good IDE, linter, lsp and type annotations could also be helpful for catching errors ... well placed try-catch blocks and assertions can be useful, too.

Bob_Li Lisa Actually the problem is, I don't just want to catch errors, But catch them all and appned to a list so that I can send them as a feedback loop to the LLM which generates manim code in my project flow. So, basically I'm looking for something that will help me check for all the errors in the file and just dump them into a list.

Yash Thale what kind of animation are you making in Manim?

Yash Thale seems pretty ambitious. The explanation part feels ok, given the current state of AI, it can probably generate a good explanation of a text document. but creating a Manim animation that goes with it is a pretty big task. I'm not sure if LLM's are trained with enough data on Manim animations to even produce a simple animation that works...

Yash Thale keeping it in Q&A would probably attract a wider audience and hopefully get new insights and feedbacks from other members. It could also serve as a reference for people thinking of starting a similar endeavour. Manim is usually used to generate short animation snippets. you probably have to break up the pylint result into smaller parts and ask the AI to generate short animations based on those, then composite the result, rather than expect the AI to generate everything in one go. AI tend to lose focus and hallucinate if fed complex prompts .

If your final aim is to generate a Manim animated video of code debugging in the 3Blue1Brown style, you could go ahead with your outlined plan. You'd probably get imperfect results, but you would get a rough code or prototype of what you want and use it as a starting point. i would suggest actually learning Manim so you could debug or even create the animations you want. You get more control and faster results this way. depending entirely on AI is a painful and expensive venture. and AI is really not big on dependability and repeatability.

if you're going to create an AI-powered Manim animator app, you would probably need to train one yourself. relying on general purpose LLMs would be too inefficient.

Bob_Li ---------Final goal ---------- Not exactly, my final goal is to provide a explaination + A visual explaination for people who like learning visually. So, the basic idea is bridging the concept to content gap with this. ------------ plans and what I have tried with LLMs till now --------------------------------------- I tried using LlaMa 4 for the manim code part, it was actually kinda not that good, altho it's visual reasoning capabilities will help me with the written text video script. I will probably use Mistral or some other for the actual code part. And learning manim part is what I'm kinda skipping for now so that I can focus on getting atleast a shabby mvp ready.

Ausgrindtube Yeah kinda, I'm making AI write a mnim code for animation and this is a lil feedback loop I'm creating for the coding part.

Ausgrindtube Again, not exactly vibe codingbut a educational tool to be precise

Lisa damn 😶, I'm not that good at Ai/ML to train my own model tho 💀

Bob_Li I'm making a system where I would just give any concept name to it as an input and it'll explain me the concept textually as well as by making a explainatory Animation in manim.

Bob_Li Yeah, That kinda is a challenge, It can definitely write snippets sooo..., I hope this feedback loop kinda improves that a bit to make atleast running animations. I guess the pylint Score feature will be helpful here, what's your take on this ? (Cuz idk why you seem to be a really experienced programmer to me the way you answer my questions and each time your solution works for me...😅)

Bob_Li Hop into messages bro, I guess we should discuss this. I already have a solution that might work for that issue. This feedback loop is like a "let's be sure that it runs" thing for me.....😅

Bob_Li Yes, That's why I have broke the task into some parts, node_1 -->> Searches raw knowledge related to the topic (web scraping), node_2 -->> Based on the raw knowledge writes a explaination. (shown to user) node_3 -->> Writes a video script for the explaination with each and every minute detail about the animation. (LLaMa 4 Maverick) node_4 -->> (A coding specific LLM) Writes the actual manim code. node_5 -->> A small lightweight LLM that checks if the explaination and script are strictly based on facts collected in step 1 and not hallucinations of the respective LLM. node_6() -->> This is the node about which this question is, this will be the code check node that will act as a feedback to coding LLM. (Note: I'm Using only Opensource LLMs for this project(I'm a student and practically broke lol 😅), Please share any suggestions you have bros.)

I'll share the system design If you want. but please don't copy paste my project 🤡😂