OK Google. Talk to TimeSeries

How cool would it be if we could build an assistant that could actually help people and integrate with Mendix. Well, guess what we did? Just that!

Not too long ago, I told you about my first experiences with building a personal assistant using Dialogflow, by Google for Google Assistant. In that blog, I also mentioned that it was fairly basic and there are loads of possibilities in Dialogflow and ‘Actions on Google’ which I didn’t explore yet. How cool would it be if we could build an assistant that could actually help people and integrate with Mendix. Well, guess what we did? Just that!

The first idea that came to mind was to come up with a personal assistant for Mendix World 2019. It would provide information about speakers and sessions. We decided to add some additional information, about Mendix and the Hackathon for example, and started building.

All the responses!

The way Google Dialogflow works, is that you can specify what kind of response you want to give the user. For example: a card with an image of the speaker would be great. Static information could easily be entered via Dialogflow, but we wanted to use our Mendix app to be more flexible in what we would return as an answer. It turned out that every type of response is a different JSON structure we had to send from our application. Imagine the complexity when combining responses (a ‘Basic Card’ with ‘Suggestion Chips’ for example). We had some work to do! We created all the different kind of JSON snippets and mappings required for our intended assistant. That was quite some work and had some element of trial and error to it: sending a response and hoping it would show up correctly in the Assistant Simulator.

Review by Google

As we were actually intending to release this Assistant to the public, we would have to go through Google’s rigorous review process. They state it can take up to 2-3 business days for a response, but I wanted to see that for myself. In reality though, they always responded within one business day and even to support requests they always responded within that time frame, so hats off to Google! We didn’t immediately want to push to production, because that means everyone (in the area you released your assistant anyway) can access the assistant. Thankfully, Google also reviews beta applications. You can add your own users as beta testers and submit the build to Google as a test. The actual response from Google as to why they denied your application is pretty helpful as well, as they explain in what part of your assistant they found something that doesn’t comply with their policies, For example, leaving the users mic open without being clear to the user that you’re waiting for a response.

A change of heart

A good way in the development of the assistant (Dialogflow was getting along nicely, as was the Mendix implementation of it all), we sat down with each other, and asked ourselves the question: Why would anyone use this assistant? The information is already available on the website and Mendix is probably building an app as well. We needed the assistant to be helpful at the touch of a button. After a good brainstorm session, we found the solution: We would link all the sessions at Mendix World to specific interests and, as a user, you can specify which of those interests are best for you. Based on that information, we are able to suggest the best session for you to attend!

Conversing with Google Assistant

Since we had created all the necessary technical mappings already, we were pretty quick in overhauling our assistant. Of course, the devil is in the details, but the biggest chunk we were able to deliver rather quickly! The way the assistant works is as follows: once you start the assistant for the first time, you’ll be asked what interests suit you best. By responding with any (or more) of these interests, the assistant will save this information and immediately give you (the next available) recommended session! You can keep asking the assistant for more recommendations. The followup recommendations are random, and are meant to create your personal Mendix World agenda! We are quite happy with the end result and we hope we’ll hear a lot of “OK Google. Talk to TimeSeries” at Mendix World! Of course, we didn’t delete the old information from the assistant, so you can still ask the assistant about specific speakers, Mendix, etc.

Tech-talk

If you’re wondering how we managed to create this Assistant, what the Mendix application looks like and any snags we hit while developing the assistant, please come to our booth at Mendix World. We’d be happy to explain to you (in excruciating detail ;-)) what we did. You’re also more than welcome to join our Meetup on May 22nd at the Mendix Rotterdam office, where we’ll take a dive in the Mendix implementation of this assistant is well. Special kudos to Adnan Ramlawi and Willem van Zantvoort for their cooperation and valuable insights!

Hope to see you at Mendix World where you’ll be able to “Talk to TimeSeries” all day!

April 11, 2019
Paul Ketelaars