[Feature] Flow insights

Hi

My Shopify app extends Flow with more triggers and actions and currently processes hundreds of thousands of webhooks, triggers and actions each day — new installs from large stores have sometimes doubled or tripled our traffic overnight, which is really exciting to see, but the lack of insights into Flow is starting to feel limiting.

To continue improving my app it’d be really useful to have some insights into Flow workflows and runs as currently the API is a bit of a black box with a single mutation for triggering workflows. For my purposes the insights could be limited to workflows and runs that include triggers or actions from my app — not all workflows and runs.

Below are some features I think would help, I appreciate some of these things can already be inferred to an extent and that my examples aren’t entirely cohesive but I think you’ll get the gist.

GraphQL objects and queries
For starters, some sort of object to represent a Flow workflow and run, preferably with the ability to query them using the GraphQL Admin API.

type Task {
  handle: String!
}

enum TaskType {
  TRIGGER
  ACTION
  CONDITION
}

type Step {
  id: ID!
  task: Task!
  taskType: TaskType!
}

type Workflow {
  id: ID!
  active: Boolean!
  steps: [Step!]!
}

type WorkflowRun {
  id: ID!
  status: WorkflowRunStatus!
  workflow: Workflow!
}

Related queries are below, these can be limited to workflows that contain triggers and actions from my app.

query {
  workflows(first: 50) { ... }
  workflow(id: "gid://shopify/Workflow/123") { ... }
  workflowRuns(workflowId: "gid://shopify/Workflow/123", first: 50) { ... }
  workflowRun(id: "gid://shopify/WorkflowRun/123") { ... }
}

Insights for flowTriggerReceive
Simply knowing if any workflows were triggered would be a massive improvement, and returning both the workflow and the workflowRun would be awesome too.

flowTriggerReceive(handle: $handle, payload: $payload) {
  workflow {
    id
    steps
  }
  workflowRun {
    id
  }
  userErrors {
    field
    message
  }
}

Lifecycle webhooks for workflows and runs
If a store saves or deletes a workflow using my apps triggers or actions it’d be great to know about it, for example

flow_workflow/create
flow_workflow/update
flow_workflow/delete
flow_workflow/run_start # maybe too noisy
flow_workflow/run_complete # maybe too noisy

# maybe even more exotic webhooks like
flow_workflow/action_added

Run completion
It’s already covered by the webhooks but I want to reiterate how useful it would be to know when a run completes, especially when a run fails in a step that isn’t controlled by my app.

Deferrable actions
Currently actions must return immediately, there’s no API for actions to defer a runs execution to a later date. The built-in Wait action can do this but it’s limited to relative time value and doesn’t support variables, preferably we could use the same API to defer with our actions as well.

For some context, my app provides support for natural language timing (along with it’s own less-than-ideal solution deferring runs), so stores can say next monday at 9am or 3 days before {{resource.date}}.

Simply adding support for variables in the Wait action would be a great improvement, although with the 90 day maximum it won’t be robust.


That’s all for now, I know these suggestions would involve a lot of time and work to implement but I’m still interested to hear everyones thoughts and if there are any plans related to these features in the pipeline.

Thanks for reading!

1 Like

Hi! I’m a developer on the Flow team. Thanks for your detailed post!

For the features you’ve list here, would you be willing to share some use cases?

As for the deferrable actions, I want to make sure you’re aware of the current feature we have with action requests returning HTTP 202s. It’s limited to 36 hours, but could be helpful for some use cases.

Thanks,
Ryan

1 Like

Hi Ryan! So cool to hear from you.

For the features you’ve list here, would you be willing to share some use cases?

The main use case for all of my suggestions is to improve efficiency and reduce server load on my end without impacting the shop’s experience. My app has a lot of triggers and actions that require a fair amount of processing behind the scenes (listening to Shopify webhooks and other third-party APIs) just to be available. If a shop isn’t using certain triggers or actions, it’d be great to avoid that extra processing.

Lifecycle webhooks of some sort might solve this the best — I guess implementation details aren’t so important, just knowledge that my triggers or actions are in use or not in use.

The GraphQL queries and response from flowTriggerReceive would mostly be used as a sanity check and reconciliation, in conjunction with webhooks.

Run completion would also help with efficiency. Certain triggers and actions can “warm up” resources on my server, this helps when an action is used multiple times in the same run — but it’d be great to know when they’re no longer needed or when the run completes.

I also provide usage statistics in my app, so it’d be great to tailor these to shops based on their workflows and usage of triggers and actions.

Lastly would be simplifying my implementation and services, some of these things can be inferred but it makes the code/product much more complicated and less robust than it would be with these features.

As for the deferrable actions, I want to make sure you’re aware of the current feature we have with action requests returning HTTP 202s. It’s limited to 36 hours, but could be helpful for some use cases.

I was aware of this, but because of the 36 hour limit and the way attempts are retried during that period, I saw it more as a failure case than a true method for deferring an action. Ideally, I’d be able to pause the run entirely and continue it at any point in the future.


I currently work around the lack of insights by letting stores disable features within the app, but this isn’t ideal and has caused confusion. Preferably, if a shop wants to use a trigger, they can just create a workflow and add it — without needing to open my app to enable functionality.

Even with the option to enable or disable features, it’d be nice to show shops a warning when they go to disable something in my app, e.g., “This trigger was used X times in the past day — are you sure you want to disable it?”

Thanks again!

We appreciate these insights!

If a shop isn’t using certain triggers or actions, it’d be great to avoid that extra processing.

For triggers at least, there is a lifecycle HTTP callback that can be configured. More details are here.

If I’m understanding correctly, a similar thing would be useful for actions so that whatever pre-requisites for your server to respond an action request during a workflow run can be set up?

Run completion would also help with efficiency. Certain triggers and actions can “warm up” resources on my server, this helps when an action is used multiple times in the same run — but it’d be great to know when they’re no longer needed or when the run completes.

By “no longer needed”, are you meaning so that you can do operations like cleaning up caches, unsubscribing from events, etc? And these operations are freeing resources that are not useful to other workflow runs?

Thanks!