Improve manuals for graphql to better support context and solution quality in LLMs like chatgpt

GraphQL Admin API Troubleshooting
1

LLMs like ChatGPT have only mediocre success in giving the GraphQl instructions for a specific task.

Shopify should optimize the manual for the API and give it in downloadable format so we can feed it as context in whatever is intelligent enough.

That would help transition. Yes I tried the AI tool from shopify too but it did not give me the results I was looking for

2

Hi - thanks for your feedback, you mentioned:

Can you say specifically how it didn’t give the required results? We’re looking to improve query/mutation generation so this would be helpful to know.

1 Like
3

I find it not always very good at keeping context in replies and conversations.

If I ask something, get a result, it’s not what I need exactly, so I add some extra context, it often forgets my initial message. As it only has a context awareness from current message and it’s own last message. - Does that make sense?

Also it has often suggested deprecated queries.

Dont really have any examples off the hand right now, sorry - just wanted to add some notes to it. Will be sure to keep you updated when I meet any problems

4

Can you say specifically how it didn’t give the required results?
Also it has often suggested deprecated queries.

Right. Very often failing queries, wrong attributes that are not allowed, wrong expectations.

And the reason is that:

  • the manuals are actually not well architected for parsing with LLM
  • changes in API also are much to frequent and with very low foresight

What Shopify has to do is:

  1. render those manuals in a better way, so you can give the context of a mutation directly and have it render it properly.
  2. provide extensive overviews for dozens of cases which mutations to use in which order.
    One of these cases is for examples the complete workflow from return to refunds, usually involves around 5-8 mutations. Which mutations? Why? How they are connected? Alternatives?

It took me 1000 lines of code from shipping to refund which is very bad if you think that this has to be done basically for every single project again and again.

So you can prompt to find what is the flow.
And later prompt for the mutation to generate.

Shopify keeps the LLM guessing and the developers too.

5

I couldn’t agree more. When implementing a certain function, I am usually confused to find which GraphQL to use