Introduction
Organisations are increasingly making use of modern data technologies to generate powerful insight. This insight is only possible with a single version of the truth. Azure Synapse Analytics and Data Factory enable insight across 100+ data sources. Combing data sources enable actionable insight and meaningful data driven decisions. All of this is possible without manual intervention as automated processes can handle data manipulation, joining and much more.
With the ever increasing need to bring in data professionals to manage data processing, organisations are feeling the pinch and looking at methods to democratise data and enable business users to make use of these platforms. Providing all these users direct access to the Azure portal can be daunting and will require providing access which typically they wouldn’t have. Furthermore, Users may not be interested in actually building the pipelines, but more interested if runs are succeeding, being schedule and/or triggered in the first place. Therefore, If a solution was in place to enable users to quickly view pipeline runs, without needing to contact data professionals, they can better self serve and ensure their own data is available in the data platform whenever needed.
This is where risual come in, we’ve been working with a UK-based organisation to help facilitate data automation. The project involved deploying data warehouse technology in Azure with orchestration managed by Azure Data Factory. Our key requirement at this point was to enable users to schedule pipeline runs on demand whilst being notified on status without having to sign into the Azure portal. The solution should be accessible from a mobile device and also enable ad-hoc cancellations, schedule new runs and viewing run details.
Our Solution
Our solution to this was developing a Canvas Power App that connected to the data warehouse whilst making use of Azure Logic Apps. Between these components a user could have a view both monthly and daily for pipeline runs, whilst having the option to trigger new runs from the palm of their hand. We’ve also created a Dataverse version of this application for easy import into other environment which could handle other automation-type workloads.
The first screen of the Canvas Power App enables a user view activity per date. Additional features enable searching by month and status. Here users can quickly navigate between dates and create new pipeline runs.
On creating a new run, the user can select pipelines based on their ID from the data warehouse. Once the run is submitted, it’s picked up by automation in Azure. This automation schedules and triggers these runs. The applications also provides users with the option to cancel runs on demand.
Benefits
The application was provided to the client alongside design documentation and user guide. This explains how the solution functions enabling both administrators and systems users to operate the application. As a result, the client has observed the following benefits:
- Greater flexibility to manage and perform actions across their data platform.
- Greater visibility of up-to-date information.
- Immediate value as the simple interface enables users to operate the application immediately.
This solution is a major step forward for the client giving them full control and flexibility. Their data platform deployment has been optimised with the Power Platform to generate powerful insight.
Next Steps
If you’d like to learn more, reach out to one of our data specialists by filling out the contact us form.
Here’s a useful link about Azure Synapse if you’d like to find out more.
