Lately I began exploring find out how to use giant language fashions (LLMs) to automate information evaluation such you can ask them questions on a dataset in pure kind and they’d reply by producing and working code. Carried out all this as an internet app, I (and also you!) may check out the ability and limitations of this strategy, in the intervening time relying totally on this system writing vanilla JavaScript:

As I clarify in that article, my most important curiosity is addressing this query:

Can I ask an LLM questions on a dataset with my very own phrases and have it interpret these questions with the maths or scripting required to reply them?

After the a number of assessments reported there, I pinpointed a number of limitations that, truthfully, preclude probably the most attention-grabbing functions. Particularly, whereas each GPT-3.5-turbo and GPT-4 display an understanding of person queries and may generate correct code for varied information evaluation duties, challenges come up when coping with complicated mathematical operations and requests of sure complexity. And I’m not speaking about very excessive complexity; for instance, each LLMs may produce appropriate code to run linear regression however failed at quadratic matches, and the place simply completely misplaced when making an attempt to implement procedures reminiscent of principal elements evaluation (PCA). What’s worst is that they usually wouldn’t even “notice” that the duty was an excessive amount of for them, hallucinating code that regarded OK on a fast go (for instance, the PCA process tried to invoke singular worth decomposition, SVD) and typically didn’t even crash, but was plain unsuitable.