Data Engineering on Eifel42

Key Insights from the Microsoft AI Tour 2025

Mon, 10 Nov 2025 00:00:00 +0000

I had the opportunity to participate in the Microsoft AI Tour, and it was exciting to see firsthand the experiences shared from real-life projects, particularly the Zava demo project.

Microsoft AI Tour, Frankfurt

Here are my key takeaways:

AI will accelerate development and streamline business workflows. It can enhance software quality and security, but it’s crucial to remember that a human must always be involved in the process and make the final quality checks.
Topics such as AI governance, AI security, and data loss prevention (DLP) will be critical in creating productive environments for all organizations, including small and mid-sized companies.
Maintaining high data quality is essential for effective data processing. Data lakes comprise multiple data sources. To transform each data source into data meshes or data products is increasingly important. This encompasses areas such as event streaming and data engineering.

In my experience, every organization or team navigates a learning curve when they begin to adopt new approaches and tools in software development. I’ve noticed that while low-code solutions can lead to impressive early progress, it’s crucial to be aware that a prototype or proof of concept does not meet the criteria for a functional system according to European and German regulations. T-shaped experts will be essential in navigating this AI landscape effectively.

EU Website Evidence Collector moves to code.europa.eu

Sun, 31 Aug 2025 13:00:00 +0200

Introducing the Free GDPR Compliance Helper

The European Union has introduced an excellent resource: a free GDPR compliance tool. → Discover the tool. To use it in a DevOps pipeline is mentioned here: → Explore Automating GDPR Checks.

Hosting Update

The Website Evidence Collector has transitioned from GitHub to the EU’s GitLab-based platform at code.europa.eu. This move not only enhances long-term accessibility but also aligns with the European Commission’s commitment to open-source initiatives.

New Download (v3.7.0)
EDPS - website-evidence-collector v3.7.0 - downloads

Wishing you a successful and (cookie-free) deployment experience!

Data in Motion Day 2024

Wed, 06 Nov 2024 22:50:55 +0200

I had the opportunity to participate in Data in Motion Day 2024 in Frankfurt, analyzing stream processing from a business analysis and enterprise architecture perspective.

One of my primary takeaways is that stream processing extends beyond mere software frameworks and data management. It encourages a more profound comprehension of business operations and their specific domains. This understanding is essential for requirements analysis and leveraging data in real-time to drive strategic decision-making.

Data in Motion 2024, Ticket

Javaland 2024, Nürburgring

Sat, 25 May 2024 21:21:42 +0100

A Java Software Development Conference

Javaland Nürburgring (Eifel), 2024

In April, the German Java conference took place at the Nürburgring. It was a new and great experience. It was the first time that I did not have to plan a trip to learn about on-site software development.

It’s amazing how life can come full circle. When I was a student, I used to listen to rock music at the Rock am Ring festival. Now, I find myself in the audience at the top of the software development scene.

After a month of deep reflection and revisiting the recorded sessions, a significant trend in enterprise software development emerged:

Domain Driven Design

I enjoyed the talks about Domain-Driven Design (DDD) architecture and business views, the discussions about exemplary architecture, and the thinking in the modules. Since I read Eric Evans’s Blue Book for the first time, Domain-Driven-Design has become the de facto standard for mapping business needs into modern architectures.

Behavior-Driven-Design (BDD) is a theoretical concept for me and a practical tool I regularly incorporate into my work.

Like every methodology, both approaches have a tradeoff between technology, business needs and values, and costs.

Event Sourcing

The talks about event sourcing, particularly Kafka, were overcrowded. I like the ideas for event sourcing and stream processing considerably (Apache Kafka, Apache ksqlDB, Apache Flink, etc.). However, using it in a productive environment still has a high learning curve and significantly influences the software architecture.

Green Software Development

Green software development is not just a trend, but a necessity in the face of increasing energy consumption from technologies like cloud computing and AI systems. By prioritizing energy efficiency, we can ensure that energy savings become a cornerstone in developing software systems, such as microservices running in the cloud, thereby contributing to a more sustainable and cost-effective future.