Your CIO and executive team made a decision to move your servers to a tier 2 or 3 data center such as ViaWest, OneNeck, Google, etc. You also have to consolidate, …
You start waking up at night, sweating and pondering “How would I migrate all data from all the heterogeneous systems to the new environment without down time and without interruption of operations?”
- Marketing wants access to their trends and analytics
- Operations want access to inventory management and process control
- Your data scientists, statisticians and R&D team want access to it all: (long term, short term, raw, structured, unstructured, reduced and unreduced, the internal and external data lakes.) In other words they want the lake, the reservoir and all streams
- And there is Bob. The genius with over 200 patents and a million CAD files.
Being the smart SA and NA that you are, you start to wonder; “Perhaps our people, you don’t understand. what …”.
Someone whispers; “ETL, file replication, backup, file synchronization, tape, FTP, copy, NFS, Disk copy … can’t you do that?”
Biting your tongue you utter “ETL? There are only 24 hours in a day. FTP? NFS? No! No! What about the Carrying capacity? Capacity utilization ?”. In situations like this, it is easy to feel you haven’t been heard at all.
Sitting back in your office chair you are frustrated. Ironically, your workspace is surrounded by loot from Red Hat, USENIX, LISA, SANS, ORACLE and MICROSOFT giveaways from the last few conferences. On the wall, you glance at some of your heroes’ posters: Bill Gate, Jobs, Richard Stallman, Bill Joy, Thomson or at Vinton Cerf’s fourty year old paper that paved the way to all of us and you realize how this is affecting your personal morale. You beg the posters for help to no avail. Not feeling heard is discounting, causing you to wonder what else might be out there. You begin to browse to monster.com, dice.com and a few others. But as a very loyal employee you decide to try more convincing methods.
You start to write on your yellow notepad:
- Max bandwidth =
- Total storage =
- Database
- Database schema
- DBSIZE
- ROWSIZE
- Sustained I/O
- DISK I/O
- Parallel I/O: controllers?
- Data migration
- Load
- Unload
- Testing
- ETA = ?
- Cutoff date?
- dt= (days*24hours *60 minutes * 60seconds)* (1-y) (y >0 < 1)
- if freq(y): ….
- Then Cutoff date = dt * …
- ...
Eureka! (you do not actually say that).
Then it hits you: “The speed of light! What if? What if …”
The engineer in you starts to think and reason takes over emotions.
Well, relax and put your critical thinking hat on. Your exec team is competing using value streams:
- Operational Excellence
- Product Innovation
- Customer Intimacy
They need to excel in at least one (or even two) of the three value streams to win.
No matter what value stream that your company uses to compete in the market place, the staff needs to leverage data for decision making and you my friend, have both the keys and the responsibility to move, protect and make that data available all the time.
So don’t send that resume yet, as an SA, NA or DS you are more than a scorpion, you are more than a hero. You have a huge data migration and a shift to take care off.
You are essential, pro-active and innovative. You have a huge responsibility and the company’s success depends on you.
Now, think about another solution to move data securely, efficiently, cost effectively and to reduce both the ETA and dt.
You know that you have many options to choose from.
Choose wisely grasshopper! Choose wisely.
–A
Share this Post