Syntax Error when creating nested list

[KonneMu]

Hi Elinteger! Great work and thanks again from the BVL!

We get a little error when runnning your code, I think its no big issue and we will handle it, but maybe you got a quick answer for this:

WARNING:root:Error type: SyntaxError, Message: closing parenthesis ')' does

not match opening parenthesis '[' on line 1 (<unknown>, line 4)

WARNING:root:Not fully correctly formatted output by LLM. Check prompt,

value has been lost! This was the LLMs answer: ["Cereal", "/", -2, "/",

"Category"],

["Rice", "Rice", 0.1, 0.01, "Check again."],

["Corn", "Maize/corn", 0.1, 0.01, "No Note."],

["Lentils", "Lentils", 0.1, 0.01, "No Note."])

and this the cleaned answer: [["Cereal", "/", -2, "/", "Category"],

["Rice", "Rice", 0.1, 0.01, "Check again."],

["Corn", "Maize/corn", 0.1, 0.01, "No Note."],

["Lentils", "Lentils", 0.1, 0.01, "No Note."])]

I think the error is pretty clear, the LLM is producing a uncleanly formatted nested list out of the provided data. Probably one would have to tighten the rules for producing these lists?

All the best
Konstantin

[Elinteger]

Hi there Konstantin!

Yea it throws an error due to the LLM adding in a ")" at the end of its output.

[["Cereal", "/", -2, "/", "Category"],

["Rice", "Rice", 0.1, 0.01, "Check again."],

["Corn", "Maize/corn", 0.1, 0.01, "No Note."],

["Lentils", "Lentils", 0.1, 0.01, "No Note."])]

chipRAG expects the LLM to either give out an actual python-list of lists or just a couple python-lists divided by commas. So the output we strive for should either be [...], [...], [...] or [[...], [...], [...]]. Both cases are fine as the "clean up" code right after prompting the LLM turns single, comma seperated lists, into a nested list.

Its extensively specified on how the LLM should do it's output in the prompts, sadly it still doesn't listen to that every time.

The options you have to fix that are

1. Try to specify the prompt even more, however I believe you will always run into problems with the current model at one point or another unless you completely re-engineer that prompt.
2. Try to adapt the temperature of the model thats being used.
3. Try out different models, though, unless you have added any new models, I believe its already using the best available LLM you got.
4. Brute force it, by adding more steps to the cleanup. In this case just stripping the LLMs answer of all opening and closing brackets (( and )) will work. Not super robust, but should do the trick in this case.

Let me know how it goes!

- Eli