No API, no problem
Organisations typically work with a variety of web applications which might lack a web API for easy integrations. Government portals in particular are often the most tightly fenced off and secured. This doesn’t stop our team from building connections to these kinds of platforms. When one of our clients called on our Ledger Labs team to build a tool that would be able to continuously retrieve data from a key government platform, we set to work.
Dashboards and platform integration
The main task for our team consisted of building a fast, always-on integration with a particular government portal that would allow our client to receive their data continuously. The end goal was to use this data for preparing monthly sales invoices, based on the figures published on the portal. In addition, we also delivered a Microsoft Power BI dashboard that gave the client more comprehensive insights than the original platform itself.
Connecting through open-source technology
We built an integration using largely open-source resources. The only licensed tool was Microsoft Power BI, which was our customer’s required platform for data visualisation. All other technologies used for the actual data orchestration between the portal and Power BI were open source.
The integration we built consists of three flows:
- Hourly data retrieval
Every hour, people log on to the government portal. For this purpose, a Dagster Orchestrator is deployed that communicates with a Selenium browser and a Vault for user credentials. Each execution reads the data that was created or modified on the portal since the previous execution (one hour earlier). The result is a structured database that makes the data easily available.
- Daily data upload
Three times a day, data is uploaded via an on-premise gateway to a Data Lake in Azure. Once the data is in the cloud, end users can consult it in a PowerBI report. - Monthly invoice preparation
Every month-end, the data is interpreted and flat rates are recorded in the invoicing software. The numbers on the sales invoices are thus directly linked to the data read out on the government portal.
Making automation pay off
Automating the connection to the government portal has streamlined the entire invoicing process for the client. Sales invoices are now prepared automatically based on the data from the government portal, eliminating manual work and reducing errors. Our client reported fewer complaints from their own partners and end clients, as invoices are both prepared faster and more accurately. When questions do arise for internal decision-making, the Power BI dashboard provides guidance with clear, detailed explanations.
From legacy ERP to modern tech stack
When international organisations switch ERP or CRM software, success hinges on more than just technical execution and smooth transitions and handovers. Beyond the typical technical hurdles, there’s also the human side. Getting teams to embrace and adopt new tools, it’s important to emphasise clear benefits while keeping migration disruption to a minimum. Primoris, an ally in advancing food safety and product quality across the agrifood sector, understood what was at stake when they decided to move several country branches to Odoo as their new accounting platform. Our Ledger Labs team handled the full migration across multiple European divisions, ensuring a seamless file transfer and smooth transition for Primoris teams.
Common pain points for ERP migrations to Odoo:
- Technical requirements for connecting legacy software with Odoo must be carefully assessed.
- The full history of each file or dossier has to be migrated.
- All settings and configurations – such as general ledger accounts, customers, suppliers, deductions and assets – must be preserved.
- Migrations must occur on schedule (e.g. mid-financial year) without disrupting existing planning.
- The file transfer process has to be fully automated to ensure a fast, scalable and seamless transition.
Effortless data migration, one country at a time
We developed a custom migration module that met all technical requirements while ensuring a seamless software transition. This tool creates a direct, one-to-one replica of any existing accounting environment in Odoo. Primoris French country team was fully migrated in 45 minutes. This minimal downtime allowed local file managers to continue working without losing any historical data or configuration settings. By handling the entire migration process, we freed up Primoris French team to let them focus on what matters most: delivering added value to their clients.
The success of this initial migration for one country convinced Primoris group CFO, Ellen De Gendt, to expand the rollout. “The productivity gap between countries that have and haven’t yet switched is striking,” he explains. “Our French team transitioned effortlessly and can now focus entirely on their added-value work. At the same time, teams still on the old systems are stuck with repetitive, manual tasks. That’s why we’re fast-tracking our Odoo rollout to every country.”
The accountancy migration was so easy that we’re fast-tracking the rollout across every country.
Planning to migrate your accounting firm’s environment to Odoo? With the module we developed for this project, switching is a piece of cake. Visit our Odoo accountancy migration overview page to see exactly how it works
Planning to migrate your accounting firm’s environment to Odoo?
With the module we developed for this project, switching is a piece of cake.
Visit our Odoo accountancy migration overview page to see exactly how it works
Avoiding data mismatches
Maintaining data consistency across multiple systems is challenging, but essential. When data conflicts between systems, you can expect confusion and errors leading to wasted time and causing frustration – especially when you can’t immediately pinpoint where the explanation lies. The best approach is to decide which software should be the authoritative ‘single source of truth’ for each type of data. Once you’ve identified where each data set should live, you can start connecting other systems to this primary source and start sharing information, for consistent data flows.
Building a flexible sensor to connect all tools
Our Ledger Labs team built a connector that allows primary data sources to push updates to all linked downstream tools at regular intervals. Using the leading system’s REST API, it checks for new ‘events’ every minute. When changes are detected, this sensor triggers the appropriate processes or so-called ‘jobs’ to keep information synchronised across all platforms. In this specific implementation, customer and project master data are synced every minute to both the client’s accounting software and CRM system. In addition, the project managers receive email notifications whenever there’s a significant update to the projects under their purview. The real strength of this sensor or connector lies in its flexibility: our client is able to define virtually any action, trigger or notification to respond to any type of event triggered by a data change.
To build this connector for our client, we chose to use Dagster. Its built‑in sensor functionality proved to be a perfect fit for our client’s needs. Dagster is an open‑source data orchestrator that’s free to use for organisations that are able to manage their own infrastructure. It also comes with an intuitive web portal that allows you to analyse the performance of your connections and data pipelines at a glance.