Useful things I know about building data platform in Azure and Power BI
During the Covid pandemic we at NHO (The Confederation of Norwegian Enterprise) have performed surveys with our members, which are Norwegian enterprises. We have asked them how business is going, if they fear bankruptcy, how the income is, and how they look at the next months going forward. Media and others have wanted this data […]
We recently started using Workplace (by Facebook) at work, so I wanted to check if I can add some data from our data warehouse as posts in groups. This turned out to be very easy, but worth a post. To set it up you go into workplace and settings for the group you want to […]
I presented a session on SQLSaturday Oslo last weekend and can share some of the key takeaways here. This is by no means a complete comparison of every feature of these techniques, but show some of the strengths and weaknesses and can be a start when considering what tech to use in your Azure Data […]
At work we have a weight loss competition that was in full progress when I started there in June. After the summer vacation it’s time to start thinking about beach body 2020 and start again, but we received some scrutiny in the BI department that this was shown on a TV screen as an Excel […]
SQL Saturday Oslo is approaching and I’m giving a session where I compare SSIS, Databricks and Mapping Data Flows for data transformation in Data Factory. One of the problems I got when setting this up is that when running SSIS in Data Factory you must handle the Integration Runtime and start (and stop) this efficiently. […]
If your organization doesn’t have enough data to require Azure SQL Warehouse with Polybase loading from data lakes, you might have observed that loading much data with Azure SQL databases can take some time. Today I show an example of how to use Databricks delta together with stored procedures to speed this up. The case […]
I’ve been struggling a bit with creating a new blog post lately. I have built upon the blog post where I set up a way to call the Strava API to collect some data with REST APIs. I wanted to process this with Databricks, but have met some problems. They are related to how Data […]
Azure Data Factory has native support for copying data from REST APIs. There are some pagination rules supported (listed here), but what to do when the API you are using does not support this? In my example I do not have any information in the header or body of my requests, but rather use the […]
I recently received a letter in my mailbox. It was sent to our address, but to someone who lived there before us, and we moved in five years ago. To make matters worse, it was sent by a private hospital and seemed to have some weight. How does this happen a year after GDPR? I […]
This post gives you everything you need to set up a pipeline in Data Factory so that your Power BI datasets can be refreshed as soon as the data is available. It builds on this post where we have set up some Azure Functions to use the Power BI APIs. The obvious reason for setting […]