Coming back to that PoC I talked about last week, one requirement was to display data in real-time. The data comes in as 22 different pieces of information every 40 millisecond. This translates into processing 25x22 (550) pieces of information every second.

That could be a problem. Imagine having to do a SELECT statement, display the data on the screen, and do that 25 times per second. How long does it take to get the data, format it and display it on the screen? Would I fall behind because of that? I could create a multi-threaded application where one thread reads the data and another displays it.

Here's another way to look at it: How does the data come into the database? Is it coming it every 1/25th of a second? How do I select that last piece of data? What if I execute my SELECT statement just before the data comes in? That means that I get nothing back and have to re-execute my SELECT. If the data comes in using the TimeSeries Real-Time Loader (RTL), I could have RTL send the data directly to the application in addition to putting it in the database. If I don't use RTL, I could consider using the Change Data Capture API to read the logs and process the data that way...

Looking at the implications of a real-time approach is all good but the right question was not yet asked: Why do you need to display data in real-time?

It comes back to requirements and this is one example why it is important to involve database experts in the early stages of application development. Database experts should be involved from day one to review the use cases and detect the database usage implications. Unfortunately, database experts such as DBAs are often involved only when it comes to deploying the application.

Coming back to my "real-time" problem...

Could I tolerate half a second delay? One second delay? that would mean one or two SELECT per second instead of 25. It's a little bit like saying: Do I want to rad a disk one byte at a time or one block at a time...

In my case, the requirement turned out to be to replay data as if it was in real-time. Quite different from the original request. Just to show what a difference you can make if you get involved in a project early.

Till next time...