Data in Motion and Data at Rest

Narrator: What is data in motion, and what is data at rest? More to the point, what’s the difference?

Dud: I’m having a nap?

Pete: A nap?

Dud: No!

Pete: Oh, bollox, must I?

Dud: Go on, Pete, you are the expert!

Pete: Bugger! I can’t be arsed with this juvenile nonsense.

Dud: Off you go!

Pete: Okay! Data in motion is data that is moving. It’s a level of abstraction designed to ensure that even thick people can understand what’s happening. It’s the equivalent of a can of coke with the printed instruction “this side up”.

Unsurprisingly, data in motion refers to data that is actively and not passively moving between systems, applications, or locations. This can include data transmitted over networks, streamed in real-time, or processed in transit.

Dud: Interesting.

Pete: It’s like when Roy Orbison died, and they announced it on the tele alongside clips of him performing. At the time, my daughter asked us what had happened. Of course, we told her that Roy Orbison had died, to which she replied, he can’t be dead; he’s still moving.

Dud: Hilarious, Pete.

Pete: Glad you liked it, Dud.

Dud: But, Pete, who defines this data at rest and data in movement stuff in this stuffy way?

Pete: People overseas who mangle the English language, Dud.

Dud: Americans?

Pete: No, just dumb arses from all over the frigging world.

Dud: God! It’s a personal pronoun’s world, isn’t it, Pete.

Pete: Well, okay!

Dud: And so say all of us!

Pete: Anyway, data in motion is about many things. Let me count the ways.

Dud: Thank you, Pete.

Pete: Data in motion is about streaming data. Real-time data flows from sensors, data governors, social media feeds, or online transactions. You know, the data funnel you chuck at Kafka and then marvel at as it chokes, complains, cries and dies.

Dud: Metamorphis was a precursor of ETL. Pete, “All language is but a poor translation.”

Pete: Data in motion is about data transfers: Data sent from one database to another or between cloud services.

Dud: Walking the silent streets of the digital city.

Pete: Data in motion is about network traffic: Information being exchanged between devices over the internet or some other network.

Dud: Data is everywhere, Pete, even when it’s not bits and bytes.

Pete: Dud, data in motion is data that is moving.

Dud: No shit, Sherlock! You must admire the genius of IT people who can massively overstate the obvious without skipping a beat.

Pete: It’s about managing moving data crucial for applications like real-time stupidity, monitoring of rubbish and responsive systems, where timely access to speculation can drive immediate falsities and inactions.

Dud: Cool! And what about the soporific data? The snoozing data? The reclining data?

Pete: Data at rest, Dud?

Dud: Yes. For example.

Pete: Data at rest refers to inactive data that is physically stored in a database or storage system and is not actively moving across networks or being processed anywhere. This type of data can include files, documents, databases, and backups not currently used in real-time applications.

Facets of data at rest include storage location, where it is stored, and how it is stored. We are talking storage systems, databases, file stores, cloud storage, and data warehouses. Which may come to mind. But data is also distributed, multiplied, and enhanced. One thing most IT people forget is that not all data is digitised. We still have analogue data stores. Document management systems have helped digitize analogue data, but analogue data still exists.

Unlike data in motion (which is transported or processed), data at rest is not affected until it is accessed and modified. Data access alone does not actually change the data.

Data at rest can include structured data, such as structured database entries, unstructured and complex structured data (such as flat files, signals, and images), and semi-structured data (such as XML, JSON, and variant files).

Proper data management at rest ensures its integrity, accessibility, and compliance. This sounds a little contradictory because a data breach is not exactly data at rest.

Also, data at rest can contain sensitive, confidential, and personal information and data. Therefore, strong security measures such as encryption, access controls, and regular audits are required to prevent unauthorized access and data breaches.

So, there you have it, Dud. Data-at-rest in a nutshell.

Dud: Any data in movement?

Pete: Must I?

Dud: Pretty please, Pete.

Peter: Oh, alright then. But not now.

Dud: Is that really it?

Pete: Yes, Dud. That’s it.

Dud: Isn’t there any more to it?

Pete: Yes, much more, but those are the essentials. The stuff that a layperson can navigate.

Dud: And the tough stuff?

Pete: I’ll save that for another day, Dud.

Martyn Jones, The Data Contrarian, Santiago de Compostela, 17th October 2024.