Panda: Performance debugging for databases using LLM agents
Published in CIDR 2024, 2024
Vikramank Singh, Kapil Eknath Vaidya, Vinayshekhar Bannihatti Kumar, Sopan Khosla, Murali Narayanaswamy, Rashmi Gangadharaiah, Tim Kraska
Debugging a performance issue in databases is notoriously hard. Wouldn’t it be convenient if there exists an oracle or a co-pilot for every database system which users can query in natural language (NL)—‘what’s wrong?’, or even better—‘How to fix it?’. Large Language Models (LLMs), like ChatGPT, seem to be a natural surrogate to this oracle given their ability to answer a wide range of questions by efficiently encoding vast amount of knowledge for eg, a major chunk of the internet. However, prompting ChatGPT with database performance queries often results in ‘technically correct’but highly ‘vague’or ‘generic’recommendations typically rendered useless and untrustworthy by experienced Database Engineers (DBEs). In this work we propose Panda, a framework to provide context grounding to pre-trained LLMs in order to generate more ‘useful’and ‘in-context’troubleshooting recommendations. Panda draws inspiration from the way experienced DBEs perform debugging, and puts a system in place with necessary components required to robustly deploy pre-trained LLMs in production for debugging. The 4 key components of Panda are:(1) Grounding;(2) Verification;(3) Affordance; and (4) Feedback. We describe the necessity and usefulness of each component and how they communicate internally to transform a given pre-trained LLM into generating in-context, actionable, useful and accurate recommendation for debugging a given database system.