• Billy's Unstructured Data Journal Entries

    I was watching an old western the other day when the villain dressed in all black was terrorizing the town, and it finally dawned on me the meaning of Black Hat!  I thought being a Black Hat would be so cool, but I was meant for more. I want to help! I guess that’s where the term White Hat came from as well, those that want to help others. I did some digging and the term Blue Hat suited me best:  an outside computer consultant employed to test systems prior to launch. I guess you could say a Blue hat is kind of a white hat too.  So…If you mix White and Blue, well that’s when I came up with Baby Blue Hat, or as I go by B4byBlu3H4t.

    Billy's Unstructured Data Journal Entries

    <ENTRY> / My room is Flooding

    I quickly realized the jar I caught it in wouldn’t be enough, IT was growing exponentially!

    I must find a way to keep up with its ever increasing size and complexity.

    I did some math. IT is beginning to seep everywhere, it’s flooding my room. One of two things are going to happen. IT will either get so big it will split and multiple, which means Mom is going to get upset OR I need to figure out how to provide a platform that grows with it. I don’t have long until it doubles if not triples in size. I need to think.

    <ENTRY> / Limitless

    I isolated some files today. They’re a common sort that people tend to work with. You know to do their jobs. IT, whatever it is, seems to want to help us. IT is showing me what can be achieved when people collaborate with this… data to do great things.

    I saw visions of molecular walls breaking down, scientists grinning and shaking hands as they eradicated disease. Then the next great play, it’s not Shakespeare, but people clapped enthusiastically. But next to that I saw quantum computing reaching to the stars with better efficiency, the next computer age dawning right there in front of me!

    <ENTRY> / The Containment Field

    So much would be possible if only I could construct the next age of storage technology, a simple jar wasn’t going to cut it. I needed something with unparalleled scale-out capabilities. And… a way to manage it all as it grew and transformed and needed space to breathe.

    But I needed something that would keep up with this explosion of data life. Something that hasn’t had a parallel since before history. Images of blueprints have haunted me since…Well forever. I understand what these blueprints are now.  I will build Isilon to keep growing… hopefully endlessly.

    <ENTRY> / Where in the world will this take me?

    I had a dream in China there were professors waiting on me. In Germany, there was a factory ready to build the most incredible things. Yet, it became a reality. It hadn’t gone unnoticed after a few quick YouTube videos were posted the world was onto my discovery.

    Maybe it was alien, a disruption to what had always been, but it offered a better way by introducing a globally accessible capability. It had grown Petabytes exponentially to Exabyte’s dwarfing all that had come before.

    <ENTRY> / Safe and Sound

    There is so much value sitting here in my room, growing elastically, what am I going to do with it all! I need to find a way to protect it so we won’t lose anything. I need a way to control access, establish roles on who can see what and how much they can use.  It’s more than a “nice to have”, it’s required. 

    Billy's Unstructured Data Journal Entries

    <ENTRY> / Sharing is caring

    IT needed to share and stretch. So naturally, I just went and invented it. I called the container ECS because it’s Elastic, and using the power of the cloud I could deliver a storage container that spanned the globe. The blueprint was there all along! I was beginning to understand what IT needed me to build.

    <ENTRY> / Global Namespace

    Something unexpected happened. I thought I was making a highly resilient storage unit for IT, but then I got a call from China and another from Germany. It seemed by a simple request, they were both able to access the data that IT was trying to share with us! Imagine the leading scientists working on something that had previously lived in my room?!?! That’s so cool! And Germany was already ramping up production having learned a lot of key insights from the data IT was pumping out.

    Just how far will this journey go I wonder?

    <ENTRY> / Are they friends? Enemies? It’s just too early to say.

    Something is happening. As IT continues to gobble up all of the earth’s data I’m seeing distinct patterns emerge. The data seems to be congregating in two separate entities. It’s almost as if they are different societies; I’m reminded of early science fiction stories like the Sand Kings by George R. R. Martin.

    They don’t seem to be antagonizing each other, but they are forming their kingdoms. It seems like they must have different needs. The Files, I guess that’s what they are calling themselves require speed and the ability to be edited on the fly by important people. They are an extroverted bunch always adapting to their surroundings.

    By comparison, the Objects (silly name I know, that’s what the scientist are calling them) seem to like things to remain a bit more constant but are happiest traveling the globe easily. If they need to change, they create a copy of themselves and then append that change. You can see the evolution of all of these objects. Whereas Files interface directly with humans, the Objects power the most pervasive application machinery and prefer to be accessed mostly!

    <ENTRY> / Harmony

    As I continue to build out the One FS to maintain all this activity it will be essential I create the right conditions for both to survive and thrive. One thing is certain, we need both to be successful. Not just anyone can find a way to tame all this unstructured data!

    Billy's Unstructured Data Journal Entries

    <ENTRY> / Clouded judgment

    IT seems to be growing too fast. I’m not sure how long I can keep providing the appropriate accommodations for our alien friend. I don't know how easy IT will be to move, the dimensions are massive. I’m investigating my options.

    <ENTRY> / Thunder struck

    On the one hand, I can turn him over to the public trust. They seem to have more than enough room to meet IT’s growth, but what about the future? Will I lose control? Will I be able to afford the costs of utilizing someone else’s space? Will this expose IT to entities and government regulations I haven’t foreseen?

    Or I could continue to build more efficiently than I have before with my private solution?

    Ugh, there’s so much on the line. The reality is I see IT’s potential; I just don’t want to lose control! Is that so selfish of me? IT represents all the data that I could ever hope to gain, IT could fuel everything, but outsourcing it shouldn’t be done without careful consideration. Some things to consider:

    • What about portability? Will I be able to get IT back…
    • Can I provide the right security, and ensure IT doesn’t fall afoul of data sovereignty?
    • What will I have to pay to access IT, or extract IT from Public cloud captivity?

    This is a really big decision, I better sleep on it. Until next time.


    Ok I know I’ve been rambling lately. There was the whole faction bit with files and objects separating. Then there were the worries around whether I kept IT locally or moved IT to a Public entity. Almost by accident, I saw it happening right in front of me, the answer all this time was One FS staring back at me.

    With support across multiple-protocols, and policy based automatic tiering I was able to take charge of the data uncertainty. While Object and File had their characteristics, they could be managed concurrently. And then I thought, what if I could harness the data from everything! Everything connected on the Internet. It would be billions of things. Internet of Things! If we can collect it all, we can make things smarter!

    <ENTRY> / Have your cake and eat it too

    I was able to move the Files to the public cloud today using a secure and policy driven approach to my data. I didn’t move everything. Obviously some files were too important and needed to be kept secretly. However, I did create a transparent extension to a cloud resource that both humans and applications would mistake for local availability (let’s not tell them shall we).

    The best of both worlds; security, cost advantages, speed, agility. I’d found a way to keep up with IT’s steady march to become the biggest thing ever seen. Just what will this mean for humanity? It’s anyone’s guess at this point. This Internet of Things will change lives!

    Billy's Unstructured Data Journal Entries

    <ENTRY> / I just didn’t realize 

    Ok, Ok, this is something new. I just hadn’t realized. There were all those files, and well just incredible amounts of objects. How could I have known? This, it might change everything. There’s so much more than the data, even more than the metadata; the analytics, they just might be the key!

    <ENTRY> / Everything has a story

    I wasn’t listening when IT was trying to tell me, maybe it was a dialect defect, but all these numbers poured out at me, and it was just so hard to make sense of them. But when I fed them into HDFS it all started to make sense to me.

    It’s not enough for the data to be available, for it to power everything. No, without a way to analyze and learn from how we are interacting with it, we’d never change anything. IT was trying to show me the way. The answer was a dichotomy: data in real-time is prized, but historical data that stretched out over the ages was king!

    <ENTRY> / Streaming 

    There needed to be a way to collect all this data as it streamed across the surface of IT’s being, and a way to harness all the data that had come before and was currently sitting in obscurity.

    I had found the answer: data was a perpetual value machine. But what would this mean for our future? Where would it lead?

    <ENTRY> / Science fiction from future to reality

    How far we have come, in just these few short weeks. I was watching Minority Report, and IT started mimicking, an incredible feat. Every day IT grows and becomes more helpful. In the movie, there were these screens that surfaced ads based on the characters on screen.

    IT proved this was child’s play, immediately showing me the cool new trailer for the Avatar movie, like a mind reader. So cool!  I wanted to know more, how did we already arrive here?!?! So soon, Minority Report was set in 2054!

    And I began to think about autonomous self-driving cars will soon be a reality and gas and oil won’t be the driving force. The answer lay in the data. We had just accumulated so much of it, and so fast, in a way that never could have been considered before. IT was the fuel for driving all these cool science fiction experiences we all thought we would never see. How cool is that?

    <ENTRY> / Indistinguishable from Magic

    Yet, there was so much more than this parlor trick, IT explained by working with the machines of the world we could bring intelligence into every aspect of our lives. Imagine every device on the internet providing information…Internet of Things! Some would call it artificial, but I think that’s pretty real! 

    But as with everything (if you’re not sensing the pattern here with IT) I’d have to keep providing the suitable accommodations for IT to keep collecting and expanding upon all this data. Sort of like Little Shop of Horrors minus the killer venus fly trap, I assure you IT is safe, just keeps growing daily.

    <ENTRY> / The internship

    You won’t believe what happened today; I guess I’ve been on the radar for a while with Michael Dell! He said my work with IT was certainly fascinating, and there was need for people like me in the new economy.

    At first I didn’t know what to say, IT has been like family, what would this internship require? Well, it turns out I wasn’t the only one who found a bit of unstructured data. Michael Dell and his crew were totally on the scene. But I had developed something interesting, and Dell EMC wanted me to join the team. I’m headed to the IQT conference. Dell and team will be showing all their data and analytics solutions at the IoT event. It’s gonna be huge!

    <ENTRY> / Reachable event horizon

    They told me of a world where we would together help IT solve all the most pressing needs of our customers through business aligned technology. I was pretty excited, because that had always been a dream, to make people’s lives better by creating something.

    IT formed the foundation for a digital transformation capability. When organizations were able to harness their data to deliver game changing services and understand their business with insights, they could become something else entirely. Using the blueprints, we had found a way to help them change embracing a new age without disrupting their existing business; it was all so exciting.

    I wasn’t quite sure what to make of IT, as we ushered in a new age. Where would we end up, it all started from such humble beginnings? In the end, it was quite the adventure, and we brought to humanity a prosperity that hadn’t been seen since the last great industrial age.