Unboxing the Mendix Data Hub

With MVP’s Maarten and Willem

As always, Mendix aims to solve struggle points when it comes to everything that is software development-related. If you are up to date on Mendix’s latest technologies and features, it is impossible to have missed the news about their latest, already announced at Mendix World 2019: Data Hub

“For developers, integration is still messy and time-intensive,” said Mendix CTO Johan den Haan. “Finding, accessing, connecting to, and massaging data typically accounts for 30 to 40% of application development time — even for low-code developers. Data Hub removes that friction.”

Being developers ourselves we noticed that, in these last few years, we are building more integrations with other applications and services and this growth does not seem to stop. Data is flying all over the place and it needs to be standardized and governable. Mendix MVP’s Maarten Bongers and Willem van Zantvoort had the chance to take a closer look at the current state of Data Hub.

What is Data Hub?

Mendix states: ‘Data Hub is the world’s first low-code integration platform’. This means that you can set up an instance of Data Hub and maintain it without having to code one bit! 

Data Hub is part of the Mendix platform and can be seen as an online place where, as a developer, you can make your data available to other applications and see what data is available. You can also add integrations from outside of your Mendix ecosystem, a feature that really adds to the benefit of this product. It provides an overview of what applications are using which integrations. This benefits the governability of your landscape immensely. In essence, Data Hub is exactly what the name implies; it is a hub where all your data meets and mingles.

Unboxing Data Hub

The big difference with other integration platforms is, apart from it being low-code, that Data Hub is tightly integrated with the Mendix Studios. That means that it is easy for a developer to build an integration with one of the available data sources. Especially, because these will become available as ‘External entities’, which we will discuss in a bit. 

How does Data Hub work?

1. Expose entities as OData

Data Hub uses OData as the underlying technology to transfer data between applications. To get started, you can expose entities from your Mendix Studio (Pro) with OData or load an external OData feed into the Data Hub.

2. Use the Hub to search, maintain and curate your assets

The Data Hub can be found at https://hub.mendixcloud.com/. You can either search for specific data or go to the catalog to get a full overview of all available services and their data. The catalog is used for searching on all the assets attached to the Data Hub and lets you maintain and document registered assets.

3. Consume and use as external entities

When the data is available in the Data Hub, you can consume it directly from your project in Mendix Studio or Studio Pro. The data becomes available as an external entity and can be used in regular retrieve activities and on pages.

4. Your landscape is updated automatically for increased insights

The landscape view lets you select a service and creates a nice overview of all apps that use this specific service. In our example, our landscape is very empty, but if  there are a lot more integrations, apps and services,you can imagine the benefit of this view.

Data Hub in practice

To be able to explain how Data Hub works in practice, we needed to get some hands-on experience with it. Therefore, we came up with the following challenge: We were going to build three apps, which you can think of as microservices.

  • Passenger app: contains passenger data,
  • Flight data app: contains flight data,
  • Flight notification app: a third app would bring them all together (and in darkness bind them) by sending a notification to a user telling them their flight’s status is updated. 

The approach we would take before having Data Hub is using published REST services in the Passenger and Flight data app and consuming them in the Flight notification app. With Data Hub, published REST services and consumed GET services can be replaced by the Data Hub. Below, we discuss both approaches so we can pinpoint the major differences.

RESTData Hub
1.Expose entity as REST service1.Expose entity as ODATA service
2.Create operations according to business need2.Search for data in Data Hub Pane in Mendix Studio
3.Copy paste, a hopefully documented, JSON schema through /rest-doc/3.Drag and drop wanted data on to the domain model and set the security in the consumed ODATA service
4.Create a JSON schema based on documentation4.Use a retrieve action of this virtual entity to get the wanted data.
5.Create an import mapping based on the JSON schema
6.Use a REST action within a microflow to get the wanted data.And setup security in the REST action.

Disclaimer: due to limitations of the Data Hub in its current form we needed to use a polling mechanism for this case. Preferably, you would want to use a push mechanism in order to get real-time notifications of flight changes.

Publishing data – standard method

Publishing a REST service looks something like this: You define a resource which can have multiple operations for different use cases, which you expose for other applications to consume the data.

Publishing data – using Mendix Data Hub

When using Data Hub, you do not publish a REST service, but you expose your data as an OData resource. This means you do not have to build separate operations for each use case, because as you will see later on, we can write XPath queries in our retrieve operation to achieve similar functionality.

By publishing this OData resource, it becomes available in Data Hub automatically. The settings in both of these approaches are nearly the same.

Consuming data

Consuming data using REST requires you to manually define message definitions or JSON snippets and create an import mapping based on the data you need. A potentially time-consuming and error-prone activity, especially when maintaining a multi-app landscape. However, creating your own import mappings and building functionality around this is highly customizable.

Here comes the big difference between the implementations. Instead of having to look up a JSON snippet (and hoping itis correct), with the Data Hub you only have to look at the available services in the Data Hub pane inside Studio Pro and drag the entity onto the domain model. These are no ordinary entities, but so-called ‘External entities’. They are an exact representation of the data on the other side in terms of a Mendix domain model. 

Automatically, a consumed OData service is created and all that is left to do is add the authentication information. 


Within a minute or two, you are able to define a retrieve activity using the newly created External Entity like the ‘Flight’ entity in the example depicted below. When the application runs, the data will be retrieved using the Odata protocol when necessary. This is very easy to do for a developer and takes away all the consuming service steps that you would have to do with normal REST.  Be careful though, do not forget to add error handling to your retrieves. In essence, it is still an API call and things can go wrong!

Final remarks

We can see from the test case we built that using Data Hub makes integrating a bit easier. Even more so for citizen developers who do not want to get lost into technical integration details or spend huge amounts of time searching for the right data.

At the moment, the Data Hub only offers ‘GET’ functionality. This means you can publish your data so it can be retrieved by other applications. The current ‘GET’ functionality only covers part of the scenarios that you would need in a bigger application landscape. We miss functionality to ‘POST’ or ‘PUT’ data from your application to other applications. 

Currently, pushing and updating data would still need to be done using REST actions or other means of integrating.Take note: these kinds of integrations will not be visible in the catalog and therefore would leave your documentation in multiple places. We have no doubt this is on the roadmap of the Mendix Data Hub. 

Sources:

  1. https://www.mendix.com/press/data-hub/
  2. https://www.mendix.com/data-hub/
  3. https://www.mendix.com/blog/data-hub-the-low-code-approach-to-data-integration/
  4. https://docs.mendix.com/data-hub/
November 19, 2020
Willem van Zantvoort
November 19, 2020
Maarten Bongers