Disclaimer: I consult for RackN, but I was not asked to write this post (or paid to do so), and it did not go through an editorial cycle with them. The following represents my own words and opinions.

Automation is a foundational pillar of digital transformation, but is it possible for on-premises ops teams to automate bare metal? Can ops teams adopt public cloud processes to advance on-premises processes? This post will define a few cloud infrastructure terms and discuss RackN Digital Rebar’s latest announcement.

Here are the RackN official announcement materials, if you’d like to skip to straight to that.

• Press Release
• Blog Post
• NewStack post, VMblog post

On-Premises Ops Teams Are Still Important!

One of the good things that came from developers operating in public cloud environments is that they developed a plethora of tools and methodologies. Many dev teams are developing applications that need to take advantage of on-premises data.

These teams are finding that data gravity is a real thing and they want to have the application close to where the data is (or is being created) to take advantage of data latency or to conform to security and other compliance regulations. On-premises operations teams are being asked to build environments for these apps that look more like public cloud environments than traditional 3-tier environments.

As on-premises ops folks, we need to understand the terms devs use to describe their processes, and see how we can learn from them. Once we understand what their desired end state for their environments are, we can apply all of our on-premises discipline to architect, deploy, manage, and secure a cloud-like environment on-premises that meets their end goals.

Instead of fighting with developers, we have the chance to blend concepts hardened in the public cloud with hardened data center concepts to create that cloud-like environment our developers would like to experience in physical datacenters.

Definitions

Before we get started, let’s define some terms. One thing I find curious is that people will posture on Twitter as experts, but when you dig into what they are talking about it isn’t clear if everyone is using the terms in the same way. So let’s set the stage with a shared understanding of Infrastructure as Code, Continuous Integration, and the Continually Integrated Data Center.

Infrastructure as Code (IAC)

This is the Wikipedia definition of IaC:

The process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools. The IT infrastructure managed by this comprises both physical equipment such as bare-metal servers as well as virtual machines and associated configuration resources.

Via Wikipedia

Automating data center components is nothing new, I wrote kickstart and jumpstart scripts almost 20 years ago. Even back then, this wasn’t a simple thing, it was a process. In addition to maintaining scripts written in an arcane language, change control was ridiculous. If any element was changed by something like an OS update or patch, changing hardware (memory, storage, etc.) or there were changes in the network, you’d have to tweak the kickstart scripts, and then test and test until you were able to get them to work properly again. And bless your heart if you had something like an OS update across different types of servers, with different firmware or anything else.

Cloud providers were able to take the idea of automating deployments to a new level because they control their infrastructure, and normalize it (something most on-premises environments don’t have the luxury of doing). And of course, the development team or SREs never see down to the bare metal, they look for a configuration template that will fit end state goals and start writing code.

This AWS diagram from a 2017 document describes the process of IAC. Please note the 5 elements of the process of IaC:

There is an entire O’Reilly book written about IaC. The author (Kief Morris) defined IaC this way:

If our infrastructure is now software and data, manageable through an API, then this means we can bring tools and ways of working from software engineering and use them to manage our infrastructure. This is the essence of Infrastructure as Code.

via Infrastructure as Code website

Continuous Integration

Another important term to understand is Continuous integration (CI). CI is a software development technique. Here is how Martin Fowler defines it:

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. This article is a quick overview of Continuous Integration summarizing the technique and its current usage.

via martinfowler.com

If our infrastructures are now software and data, and we manage them via APIs, why shouldn’t on-premises ops teams adopt the lessons learned by software teams that use CI? Is there a way to continually integrate the changes to our infrastructure will absolutely require automatically, something that kickstart never really handled well? Is there a way to normalize any type of hardware or OS? What about day 1 and day 2 operations, things like changing passwords when admins leave, or rolling security certs?

Most importantly, is there a way to give developers the cloud-like environment they desire on-premises? Can developers work with on-premises ops teams to explain the desired end state so that the ops team can build this automation?

Continually Integrated Data Center – a New Methodology for On-Premises Ops Teams

RackN is a proponent of the Continually Integrated Data Center (CI DC). The idea behind CI DC is approaching data center management in a software CI approach, but down to the physical layer. RackN’s CEO Rob Hirschfeld explains it this way:

What if we look at our entire data center down to the silicon as a continuously integrated environment, where we can build the whole stack that we want, in a pipeline way, and then move it in a safe, reliable deployment pattern? We’re taking the concept of CI/CD but then moving it into the physical deployment of your infrastructure.”

To sum up, CI DC takes the principles from CI and IaC but pushes them into the bare metal infrastructure layer.

RackN Digital Rebar – a CI DC Tool for On-Premises Ops Teams

RackN’s goal is to change how datacenters are built, starting at the physical infrastructure layer, and automating things like raid/firmware/bios/oob management, OS system provisioning, no matter the vendor of any of these elements or the vendor of the hardware on which they are hosted.

Digital Rebar is deployed and managed by traditional on-premises ops teams. It is deployed on-premises, behind the firewall.

Digital Rebar is a lightweight digital service that runs on-premises behind the firewall and integrates deeply into a service infrastructure (DHCP, PXE, etc). It is able to manages *any* type of infrastructure, from a sophisticated enterprise server to a switch that can only be managed via APIs to a raspberry pi. It is a 100% API driven system and has the ability to provide multi-domain driven workflows.

Digital Rebar becomes the integration hub for all the infrastructure elements in your environment, from the bare metal layer up. Is the end state that has been requested to stand up and manage VMware VCF? RackN has workflows that help you build the physical infrastructure to VMware’s HCL, including hardening. Workflows are built of modular component that let you drive things to a final state. Since it is deployed on-premises, behind the firewall, it is air-gapable for high security environments.

What’s new in Digital Rebar v4.3

Here are the new features available in the 4.3 launch:

• Distributed Infrastructure as Code – delivering a modular catalog that manages infrastructure from firmware, operating systems and cluster configuration.
• Single API for distributed automation – providing both single pane-of-glass and regional views without compromising disconnected site autonomy.
• Continuously Integrated Data Center (CIDC) workflow – enabling consistent and repeatable processes that promote from dev to test and production

Real Talk

Not all compute will be in the cloud, but developers have new expectations of what their experience with the data center should be. Most devs write in languages written for the public cloud. Traditional data center platforms like VMware vSphere are even embracing cloud native tools like Kubernetes. All of this is proof we’re in the midst of the digital transformation everyone has been telling us about.

Sysadmins, IT admins, even vAdmins, this is not a bad thing! On-premises ops teams can learn from the dev disciplines such as IaC and CI, and we can apply all the lessons we know about data protection, sovereignty, etc. to use new ops processes such as CI DC. It’s long past time to adopt a new methodology for managing data centers. Get your learn on, and get ahead of the curve. Our skills are needed, we just need to keep evolving them.

Disclaimer: I recently attended Storage Field Day 19. My flights, accommodation and other expenses were paid for by Tech Field Day. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event. Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

Yesterday, Dell EMC’s Project Nautilus emerged as Dell EMC’s Streaming Data Platform. I wrote this post based on the presentation we were given at #SFD19, and decided to keep the Project Nautilus name throughout my report.

I love it when presenters tell us what world they are coming from, and tie our shared past to new products. Ted Schachter started his career at Tandem doing real-time processing with ATM machines. But as he pointed out, these days there is the capacity to store much more info than he had to work with back in his Tandem days. I loved how he drew a line from past to the present. We really need more of that legacy, generational information shared in our presentations to help us ground new technologies as they emerge.

Data Structures are Evolving

Developers are using the same data structures they’ve used for decades. There is an emerging data type called a stream.  Log files, sensor data, and image data are elements you will find in a stream. Traditional storage people think in batches, but the goal with streams is to move to transacting and interacting with all available data in real time, along a single path. By combining them all these data types into a stream you can start to observe trends and do things like the ones shown on the slide above.

Since the concept of streams is pretty new, the implementations you’ll see now are DIY. There are “accidental architectures” based on kafka. Kafka is an open source Apache platform for building real-time data pipelines and streaming apps.

Project Nautilus Emerged to Work with Streams

Project Nautilus from Dell EMC Storage is a platform that uses open source tools. They want to build on tools like Spark and Kafka) to do real time and historical analytics and storage. Ingest and storage is via Pravega. Streams come in, they are automatically tiered to long-time storage. Then it is connected to analytic tools like Spark and Flink (which was written specifically for streams). Finally, everything is glued together with Nautilus software to achieve scale (this is coming from Dell EMC Storage after all), and is built on VMware and PKS. More details were to be announced at MWC, so hopefully we’ll have some new info soon.

Real Talk

Product Nautilus emerged as a streaming data platform. This is another example of Dell EMC Storage trying to help their customers tame unstructured data. In this case, they are tying older technology that customers already use to newer technology – data streams. They see so much value in the new technology that they created a way for customers to get out of DIY mode, while at the same time taking advantage of existing technical debt.

This is also a reminder that we’re moving away from the era of 3-tier architecture. There have been hardware innovations, that has led to software innovations. We are going to see more and more architectural innovations. Those who open to learning how tech is evolving will be best positioned apply the lessons learned of the past couple of decades.

How are you learning about the new innovations?

Disclaimer: I recently attended Storage Field Day 19. My flights, accommodation and other expenses were paid for by Tech Field Day. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event. Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

Taming Unstructured Data

A common thread discussed by almost every vendor we visited was the issue of taming unstructured data. Vendors are building products that their customers can use to turn massive amounts of unstructured data into information. They all told us that their customers are demanding intelligent insights that are available anytime accessible anywhere. The groups from Dell EMC Storage were no different, they are also tackling this problem.

Four storage product teams came to chat with us during SFD19: Isilon, the Project Nautilus team, a team building devops tools, and PowerOne. What’s interesting is that in addition to tackling the challenge of taming unstructured data, each of these product groups are working on the innovations to traditional storage products that enable them to integrate with products and services we usually think of as with cloud native solutions, for example Kubernetes.

I’ll tackle each of the areas that I mentioned above, and this post will concentrate on Isilon.

Taming Unstructured Data with Isilon

Isilon Systems was founded in 2001 and acquired by EMC in 2010. Dell EMC Isilon is a scale-out NAS that is run on a file system called OneFS. The team has even won an Emmy for its early development of HSMs (hierarchical storage management).

Isilon’s definition of scale out is policy-based management. Every node is independent and able to access data coherently. The files aren’t being split, but you can keep snapshots in a diff tier. Users write the policies and the system takes care of it from there.

CloudIQ (Dell EMC’s SaaS infrastructure management tool) now supports Isilon. They also acquired a tool called ClarityNow which is included with an Isilon license (as is CloudIQ) although you are charged for non-DellEMC storage.

OneFS Gets Data Closer to Cloud Compute

Isilon OneFS is also available to run with compute in the public cloud. Dell EMC partners with service providers to offer Isilon OneFS on Dell EMC metal at their co-los that are located close to public cloud providers. It’s offered as a SaaS service and is great for current on-premises Isilon customers who want to extend their Isilon implementation to the cloud for  DR, replication, or even to perform new types of compute like machine or deep learning.

But *why* would customers want to do this? If you’ve stored your unstructured data in an Isilon for even 10 years, that is a tremendous amount of data gravity. It’s going to be hard to move this data to the cloud, even if the services and tools you’d like to use are there. Isilon’s OneFS structure allows you to extend this data to other locations, and if the locations are connected via a fast pipe in a co-lo center to a cloud, you can design to take advantage of the best of both worlds.

Real Talk

This is a great example of how traditional storage product teams are working with cloud product teams to create offerings to support the customers who are writing apps and taming unstructured data. Customers realize to do that, they have to go beyond polarizing architectural attitudes like “everything cloud” or “cloud is evil”.

These customers understand that when it comes to taming unstructured data, the devil is in the details. It is still the responsibility of the architect to understand what you’ll be signing up for with any of these types of solutions. Ask lots of questions, and weigh you’re the risks and benefits to be sure this type of solution will work for your organization.

Disclaimer: I recently attended Storage Field Day 19. My flights, accommodation and other expenses were paid for by Tech Field Day. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event. Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

The first presentation of Storage Field Day 19 was Tiger Technology. They are a data management company that has been around since 2004, mainly providing solutions primarily for the media and entertainment industry.

This industry is interesting to modern storage because of their application requirements, in particular video. These applications are usually mission critical, and require high bandwidth and low latency. Because these applications are so diverse, there really isn’t a standard. One requirement they do all have in common is that they are intolerant of data loss. Think of video games suffering lag, or a live sporting event dropping frames or even pixels – these are just not acceptable performance in this industry.

The Tiger Technology team took us on the journey of how they built their new Tiger Bridge offering. Tiger Bridge is a cloud tiering solution for Windows (they are working on Linux) that brings cloud storage to current (and legacy) workflows in a way that is invisible to your workers.

Tiger Technology’s Journey to the Tiger Bridge

The customer problem that took them on their journey to create Tiger Bridge was surveillance for an airport. The airport wanted to upgrade their surveillance systems. They had 300 HD cameras with a retention time of 2 weeks and wanted to scale within 3 years to 10,000 4K cameras that would have a retention of 6 months. Tiger Technology computed that the capacity for this project would be ongoing at 15 petabytes of data.

Tackling this problem using standard file systems would be prohibitively expensive, not to mention that it wasn’t even possible to get Windows to that capacity at the time they started.  They knew object storage would work better. Because of the security implications, other requirements were no latency or BW impact, no tamper point, software only, and scalable.

If you think about surveillance cameras, you need a way to keep the data on-site for a while, then you need to send the data someplace that doesn’t cost as much to store it. But you need to be able to bring that data back with fidelity if you need to check the videos for something. These customer challenges are how they came up with the idea for Tiger Bridge.

What is Tiger Bridge?

Tiger Bridge is a hierarchical storage management (HSM) system. It installs in less than five minutes on a server. The agent installed on the servers is a Microsoft Filter Driver and sits between the application reads and writes and target storage.  Since it is integrated with the file system as a filter driver it also falls under Active Directory control, which is great for existing workloads and policies.

With Tiger Bridge, files are replicated and tiered automatically based on policies set on last access time and/or volume capacity. The agent does the tiering work in the background, so sending or retrieving the file from the cloud, even cold cloud storage, is transparent to the user.

The team focused on providing this seamless experience to applications that are hosted on the Windows platform. Since they wanted this to also work for legacy apps, one thing they had to figure out is how to use all the commands that are common in a file system that aren’t replicated in the cloud, things like lock, move, rename, etc. They also wanted to support all the cloud storage features like versioning, soft delete, and global replication, since applications written for the cloud require these features.

The example they gave of bridging cloud and file system features was rename. You can rename any Windows file, no problem. But rename isn’t available on public cloud systems, you have to do a copy. For a couple of files, that’s probably no big deal. But if you rename a folder with lots of files in it, that could be a huge rename job. It may take time, and it will probably get expensive.

Their solution keeps track of where the files are, and any changes that have been made. This solves the problem of data being rendered useless because it’s no longer associated with its original application, a common issue that brings on lock-in anxiety. Files under the Tiger Bridge control maintain a link with the file system on premises and the public cloud. Users never know if they are hitting the data on premises or in the cloud.

Check out the demo from a user perspective:

What does Tiger Technology do for users?

What this means is that a user on their laptop can use the Windows file system they are familiar with, and the agent handles where the file actually is in the background.  Administrators can make rules that tier the data that make sense to the business. It allows organizations to use the cloud as an extension of their data storage.

Other use cases are disaster recovery. Having a location like the cloud so you can have a backup of your data in a different location without having to manage another site or tapes is a very attractive use case. Since it is so easy to bring files back from the cloud, Tiger Bridge is able to handle this use case as well.

Real Talk about Tiger Technology

 I think this is the year we’re going to see a lot more solutions bubble up that truly bridge on-premises and the cloud, and I think we’ll seem them from older companies like Tiger Technology. These companies understand application requirements and the technical debt that companies are battling with, and they are finding ways to make the cloud model fit into the reality of their customers’ current realities.

The Tiger Technology presentation reminded me of something we used to say at EMC: a disk, is a disk, is a disk. Users, and applications, don’t really care where the disk they are writing to is located, who manages it, and what it costs. They care about their application being easy to use, low latency, and security. Tiger Technology has figured out how to make that old storage saying work for public cloud and legacy applications.

What do you think? Let us know in the comments!

Disclosure: I was invited by GestaltIT as a delegate to their Storage Field Day 19 event from Jan 22-24, 2020 in the Silicon Valley CA. My expenses, travel, accommodation and conference fees were covered by GestaltIT, the organizer. I was not obligated to blog or promote the vendors’ technologies. The content of this blog is of my own opinions and views.

Is storage still relevant in today’s cloud and serverless environments? At Storage Field Day 19 we spent several hours with Western Digital, and heard from ten different presenters. Did they show us that storage is still relevant?

Hardware Must Innovate for Software to Innovate

I think the industry often forgets that software innovation is impossible without hardware innovation. We’ve seen some pretty amazing hardware innovations over the last decade or so, and hardware companies are still at it.

You may be asking: how is an old hardware company able to keep up, let alone still be innovating? Well, Western Digital has 50 years of storage experience, and they are still innovating. Their heritage is highlighted in this slide.

Western Digital is looking at how to solve the data storage challenges for emerging workloads. They already have tons of experience, so they know that the data must be stored, and that more data is being created now than ever before.

More data is being created today than ever before, and it all needs to be stored so it is available to have compute applied to it. Compute is what turns the data is turned into actionable information. But there is so much data now – how should it get stored? How will it be accessed? It’s becoming pretty obvious that the old ways of doing this will not be performant, or maybe not even scalable enough.

One workload they talked about throughout many of the presentations was video. Just think about what kinds of devices that now create streams of video. IoT devices, survellance cameras, cars, the general public, etc. Much of the new types of streaming video is being created at the edge. The edge cases are so diverse that even our understanding of “edge” may be antiquated.

So is storage still relevant? Maybe not the type I came up on – SANs and NASs. But the next evolution of storage has never been more relevant than now.

Composable Infrastructure

Western Digital also discussed composable infrastructure, and how technologies such as NVMe over Fabric make composable infrastructure possible. Don’t worry if you have no idea what I’m talking about – the standards for NVMe over Fabric weren’t pulled together until 2014, and the standard became real in 2016. Also, hardware standard boards are so peculiar – they don’t use the NVMe acronym, they use “NVM Express”. This makes it hard to find primary source information, so keep that in mind when you’re googling.

What can NVMe over Fabric do for composable infrastructure? First, let’s answer why would you need composable infrastructure?

Western Digital’s Scott Hamiliton walked us through this. First of all, new types of applications like machine learning and deep learning need the data to be close to where the compute is happening. Even after considering tradeoffs that must be made because of data gravity, traditional architecture slows things down because resources are locked in that traditional stack.

Composable infrastructure takes the resources trapped in traditional infrastructure, breaks them up and disaggregates them. After that’s done, the resources can be recreated into the leanest combination possible for a workload, virtually composed, creating a new type of logical server. The beauty is this can then be modified based on the dynamics of a workload.

According to Hamiliton, Western Digital believes NVMe will the foundation of next-gen infrastructures, and that eventually ethernet will be the universal backplane. It was an interesting session, check it out for yourself below.

Zoned Storage

Western Digital is also championing the Zoned Storage initiative. This will be part of the NVMe standard. Zoned Storage creates an address space on disk (HDD or SSD) that is divided into zones. Data must be written sequentially to a zone, and can’t be overwritten sequentially. Here’s Western Digital’s explanation:

[Zoned Storage] involves the ability to store and retrieve information using shingled magnetic recording (SMR) in hard disk drives (HDDs) to increase the storage density and its companion technology called Zoned Name Spaces in solid state drives (SSDs).

via https://www.westerndigital.com/company/innovations/zoned-storage

Why does the industry need this? According to Swapna Yasarapu, Sr. Director of Product Marketing for Western Digital’s Data Center Business Unit, we’re moving into an era where large portions of unstructured data are being created. All of this data can’t be stored via traditional methods. Additionally, unstructured streams come from IoT edge devices, video, smart video, telemetry, and various other end devices. Many of these streams must be written sequentially to unlock the information the data contains.

Finally, this is an open source initiative that will help write this data in a more practical way for these types of data streams to HDDs and SSDs.

Watch the entire presentation here:

Acronyms as an innovation indicator

One way I can tell when there is innovation is when I come across acronyms I don’t know. After 3 years focusing on virtualization hardware, I found myself having a hard time keeping up with the acronyms thrown at us during the presentations.

The good news is that some of these technologies are brand new. So much for storage being old school! Plus, can you imagine what apps are waiting to be written on these new architectures that have yet to be built?

Here are the acronyms I didn’t know. How many can you define?

• TMR: tunneling magnetoresistance
• TPI: Track Per Inch (disk density)
• PZT: Piezoelectric actuator (see this earlier Storage Field Day post)
• VCM: Voice Coil Motor (see this video )
• SMR: Shingled Magnetic Recording
• SSA: Solid State Array
• ZBC: SCSI Zoned Block Commands
• ZAC: Zoned ATA Commands
• ZNS: Zoned Named Storage

Is Storage Still Relevant? Final thoughts

I think you know my answer on the questions is storage still relevant: of course! We are just beginning to create the standards that will issue in the real digital transformation, so there is plenty of time to catch up.

I’m excited that I have been invited to be a delegate at Storage Field Day 19. This is a little different than the Tech Field Day I attended in 2019, because the focus of all the presentations at this event is data storage.

I am looking forward to this because I am a storage person. My career started as a Technical Trainer at EMC, I was a storage admin for a pharma company. I went back to EMC to develop technical training, I then went to work for Dell Storage, and then Inktank (a startup that provided services and support for Ceph). I guess you could say storage is in my blood, so Storage Field Day should be lots of fun.

What to expect at Storage Field Day

Here are the companies we’ll be visiting (in the order they will be presenting), and what I’m looking forward to hearing about from them. Remember, you can join in on this event too by watching the livestream and participating in the twitter conversation using the hastag #SFD19.  You can @ me during the livestream and I can ask a question for you.

Disclosure: I am invited by GestaltIT as a delegate to their Storage Field Day 19 event from Jan 22-24, 2020 in Silicon Valley. My expenses, travel, accommodation and conference fees will be covered by GestaltIT, the organizer and I am not obligated to blog or promote the vendors’ technologies to be presented at this event. The content of this blog represents my own opinions and views.

Tiger Technology

The first presentation we hear will be from Tiger Technology. Just looking at the website, they claim to do lots of stuff. When I look at their About page, they’ve been around since 2004 “developing software and designing high-performance, secure, data management solutions for companies in Enterprise IT, Surveillance, Media and Entertainment, and SMB/SME markets”. They are headquartered in Bulgaria and Alpharetta, and since my mom was born and raised in Alpharetta, they get extra points.

Skipping to their News page, it looks like they have a new solution that tiers data in the cloud. I’m looking forward to hearing how they do that!

NetApp

NetApp talked with us at TFD20 (my blog review of that presentation). They talked to us then a bit about their flavor of Kubernetes, and the work they are doing to make it easy for their customers to have data where they want it to be. Hoping they do a deeper dive on CVS and ANF, their PaaS offerings for the current public cloud offerings.

Western Digital

Western Digital has presented at previous Tech Field Day events, and have acquired many companies who are Tech Field Day presenting alums. The last time they presented back in February 2019 they talked about NVMe, and I love that topic.

One thing I think that doesn’t get enough attention is the incredible innovation that has happened over the last several years in storage hardware. The software is now catching up, and apps will follow. So there is cool tech stuff happening on prem too, not just in the public cloud domain.

I peeped their twitter account, and they have interesting things they are showing this week at CES. Like this 8TB prototype that looks like a cell phone battery bank.  That would be a pretty sweet piece of swag! 😊

Infrascale

This will be Infrascale’s first appearance at Storage Field Day. Their website says what they do right up front: they have a DRaaS (Disaster Recovery as a Service) solution that fails to a second site, booting from an appliance or the cloud.

After storage, the biggest time I’ve spent in my career has been with data protection and disaster recovery, so I’ll be looking forward to this presentation as well. Really looking forward to hear about how this solution can included in an architecture.

Dell EMC

Since I’ve worked in storage at Dell and EMC, and I’m just coming off a tour at VMware, of course I’m excited to sit in on presentations from my Dell Technologies federation homies! There will be presentations on Isilon and PowerOne, but the one I’m most curious about is one on DevOps.

Komprise

Komprise has presented at Storage Field Day before (in 2018). They are a data management and tiering solution. At AWS re:invent they unveiled a cloud data growth analytics solution. I hope we hear about that.

WekaIO

WekaIO’s  has presented at Tech Field Day a couple of times before. They have a distributed storage system for ML/AI, it looks like they directly access NVMe flash drives. It looks like they also have a solution on AWS. So this should be an interesting conversation. I’m just hoping we don’t have to listen to a “what is AI story” before they get to the good stuff.

Minio

This will be Minio’s first presentation at Tech Field Day. Minio sells high performance object storage. One of the other Tech Field day delegates, Chin-Fah Heoh, has already written a blog post about how Mineo is in a different class than other object storage providers. I’m really looking forward to this presentation.

This post – NetApp Goes to the Cloud – is my review of materials presented at #TFD20.

NetApp’s 1^st presentation at #TFD20 was about NetApp’s cloud strategy. I was very excited to see Nick Howell (aka @DatacenterDude), NetApp’s Global Field CTO for Cloud Data Services, there to greet us and kick things off.  I’ve always known him to be knowledgeable, visionary, and a bit controversial. All of my favorite things! And I was psyched to see how he was going to frame his conversation.

Infrastructure Admin’s Journey to the Cloud

Nicks’ presentation was titled “OK, Now What?” An Infrastructure Admin’s Journey to the cloud.

He set up the history of things for datacenter admins, and how quickly they need use their existing skills to pivot if they’re going to support cloud. I liked this slide highlighting historical design patterns for datacenters.

He gave a great overview of the struggles IT Ops folks will need to go through in order to support their organization’s move to the cloud: new training, new certs, etc. It will take effort to get up to speed from a technical perspective.

NetApp Goes to the Cloud

Of course, the message was how easy NetApp makes it for their customers to get to “the cloud” using NetApp Cloud Data Services. He brought in the Google Cloud Partner of the Year award that NetApp was awarded this year’s at Google Next. To me, that makes it obvious they are doing the hard integration work to enable hybrid cloud with NetApp storage.

They’ve been at this for a few years after hiring an exec to run a cloud business in 2017, and acquiring cloud startups (Greenqloud 2017, StackPointCloud 2018). Two years later, NetApp has built a suite of cloud products that are delivered in the cloud, as-a-Service, by NetApp.

They have an IaaS offering called CVO (Cloud Volumes ONTAP), which is a virtual version of ONTAP in the cloud which allows customers to do everything they would do with ONTAP on prem plus more in the three major public cloud services. They have a free trial if you’re interested in kicking the tires. There are also two PaaS offerings called CVS (AWS Cloud Volumes, Google Cloud Volumes) and ANF (Azure NetApp Files).

They are building a control plane, that Nick compared to vCenter, called Fabric Orchestrator. It will give a global view of all data, no matter where the data resides. You’ll have oversight and management control from this control plan. This is set to launch in 2020.

NetApp Kubernetes Service

While this is great work to provide the services to make NetApp hybrid architectures possible, what can you *do* with it? Data capacity exists to host applications, and the way to orchestrate modern applications is Kubernetes.

NetApp has their own Kubernetes service that they call NKS. It is a pure upstream Kubernetes play, and they support the latest release within a week. It has been built to provision, maintain, and do lifecycle management no mater the cloud on which it runs.

Real talk

From everything we were shown, if you’re a NetApp customer you have lots of opportunity on which cloud to use as you build a hybrid and/or multi-cloud strategy. You have a a cloud organization that understands your fears and pains, and they are working to make cloud as easy as possible for you.

NetApp seems to have the right team and attitude to make multi-cloud a reality for their customers. They’ve built a cloud team from cloud native veterans to drive this strategy. They seem to be very intent on shepherding traditional operations teams into the new cloud native era. Will this be enough to span the digital transformation gap? Only time will tell.

Last month I accomplished my dream of becoming a Tech Field Day delegate for #TFD20. Because I left my job at VMware in order to launch Digital Sunshine Solutions, I finally no longer work for a vendor and I qualify to be a delegate! This post is a reflection on the differences between being at a vendor and hosting Tech Field day and being a delegate.

Tech Field Day history

For those of you who don’t know, Tech Field Day is a roving event run by Stephen Foskett. 10 years ago, when we were all figuring out what blogging and podcasting meant to big tech companies, he had the vision to take influencers who were talking technically and strategically about products on their personal blogs and podcasts right to the vendors. This gave vendors the opportunity to explain the products and processes, as well as meet this new type of advocate/influencer. Stephen paved the way for enthusiastic, independent influencers the same recognition as analysts and press have always received. Smart vendors welcomed his travelling crewe into their inner circle.

My first time as a delegate

The reason I’ve never participated before: I’ve always worked at a vendor! You can see from my Tech Field Day Delegate page that I’ve participated as a vendor and blogger since the beginning of Tech Field Day. I’ve been responsible for organizing and hosting as a vendor and let me tell you that was no small accomplishment at the time!

Experiencing Tech Field Day as a delegate was exponentially more challenging than following or even hosting as a vendor. Most days we needed to be downstairs before 7. So I was up early to go to the gym and put on makeup. I hate wearing makeup, but my good friend Polly has been playing with a YouTube channel and let me know that if you’re on camera, you need makeup. She is probably right.

We traveled to several vendors a day, hearing their current pitches. Some were amazing, some could have been better. Everyone was very nice though, and treated us like VIPs. After a full day of presentations (we went from 7 – past 5 every day), there were dinner and socialization activities.

I have known most of the other delegates for a long time (decade even). Talking about the technical and business challenges brought up by the vendors really did bring us together in a community for the week.

What’s in it for vendors

Since I’ve worked for a vendor, I know how hard it can be to secure the funding to bring Tech Field Day to your company. In case you had any reservations, let me put your mind at ease: every single delegate is very keen to hear, understand, and discuss what you’re presenting. There was so much experience in our set of delegates that we had some very vigorous discussions about what you presented. I’m just now getting around to writing blog posts, because I needed the time to reflect and research a bit before I put pen to paper.

The food and swag all were nice, but we were honestly most interested in what your speakers had to say. A couple of the presentations were a little rough, and we found out later that the folks presenting were tapped at the last minute. This is no disrespect to those presenters, but vendors you really want to ensure that you have your guru in the room. Even if they are a little rough, just coach them on what not to say. Let them get up there and geek out. Having folks present that are super safe because not as comfortable with material as they would have liked, or worse sticking to a script is very frustrating, I know this can happen when someone is asked to cover at the last minute. It just leaves you with this feeling that the really good stuff is missing.

The Tech Field day event has always been such a good blend – mixing curious, experienced techies with the product people who want feedback and input to their product strategy. If you have a new message or launch you would like to test, Tech Field Day is a great vehicle for that.

Participate in the Tech Field Day Community

There are so many ways to participate in the Tech Field Day community! To start with, you can watch all of the events live online from the Tech Field Day website. If you’re a vendor, you can become a sponsor and have the delegates live at your location. If you’re an independent techie, maybe one day you can also live the dream.

As the end-of-year conference dust settles, it’s becoming clear that we’re finally experiencing the long-heralded digital transformation. We have cloud, we have hybrid cloud, we have edge, but the technologies have improved to the point that it’s possible now to choose how to implement what has become known as “the cloud”.  As always, my question is what does this all mean for ops?

The Amazon Web Service conference re:Invent is this week in Las Vegas. I’ll be there to chat with people, trying to figure out if what I think is happening matches the reality of what’s happening in organizations. As part of my prep, I wrote this blog post. It is meant to be a bit controversial. My end goal is to find the reality of our current situation, find ways to acknowledge the hype and with real community building move past it. Who’s with me?

Simplified (and slightly dramatized) cloud history

The original target market for the public cloud providers was developers. Public cloud providers understood that developers were stifled by tedious IT rules that many times seemed like process for process’ sake. And if we’re honest as ops people, we hated having to enforce these arcane rules just as much as the developers hated dealing with them.

The cloud providers were smart. They built the scalable architectures that ops couldn’t (or wouldn’t, or weren’t allowed to) build. Then they went after the developers, whispering in their ears saying “we know your IT team is difficult, we know they suck, come build with us. We will give you the environment you want. No more putting in ticket requests with a 4 week SLA, or having to wait forever to be assigned more resources. Come to our cloud, and you can just build your code, run it and as you need resources, we will provide them. We got you.

Devops is born

Development teams building out cloud native applications (perhaps containerized applications) figured out pretty quickly that they needed operational expertise to get the job done.  And devops was born. They needed someone who understood networking, vms, containers, security, as well as the application itself. So a new flavor of sysadmin was born: the SRE. Or maybe you call them a Platform engineer. Whatever you call them, they are a ninja that understands how to build and manage a cloud environment that hosts the new applications being built by develoers. Also, they are able to collaborate and work with developers.

These folks bring all the on-prem ops knowledge and lessons learned from managing traditional apps, but with a twist: they can no longer touch the infrastructure, many times they can’t even manage the VMs. Think about the pressure there: they must deeply understand what the cloud provider is offering and match it to their application’s requirements. And those services (including infrastructure) can be changed or even cancelled by the cloud providers change at any time. These ninjas must understand the architecture of the application and be able to adjust that design if the cloud provider changes up the offering.

Oh they are also saddled with all of the data hygiene requirements of traditional applications: backups, restores, security, access, archiving.

What devs want vs what they need

When we think about why IT teams were notoriously so hard to deal with, part of the reason is because supporting enterprise apps is really hard. It turns out maintaining data, protecting it, backing it up having it always available takes real strategy and work. Add in dealing with latency issues and dealing with privacy regulation issues and the complexity only increases.

Not that we needed the crazy process we’ve seen over the last several years. Our charter as ops is to more than just keeping keep the datacenter lights on, it is to provide devs with the tools they need to create apps that support the business. Somehow that got lost, and the cloud providers pounced on the opportunity to grab all of those workloads.

Reality of running workloads sets in

In addition to the complexities of datacenter hygiene, there is the question of whether all workloads belong in the cloud. Some reasons to consider keeping applications on premises are privacy, latency, and data gravity.

The principle of data gravity proves true here: as data grows in mass, it is more likely that applications will be created close to where the data physically resides. And while it is true that cloud applications and even IoT devices are creating masses of new data, in many cases it is augmenting and/or blending with data from 30 – 40 years ago, and that data resides in big metal storage arrays and even mainframes.

So the question becomes: do all applications belong in the cloud? Can a flexible, elastic (cloud-like) environment be built on-premises? Is this an either-or decision, or can IT ops use all of the available options to build and maintain the best infrastructure for any given application?

Digital Transformation is here

The digital transformation that we’ve heard about from analysts and marketers for the last several years is finally here. It’s real, we’re in the midst of it, and we’re starting to see some real battle lines being drawn between traditional on-premises vendors and cloud providers.

Can cloud providers figure out how to deal with the latency issue? I’m sure we’ll hear about that this week when AWS reveals their progress on Outposts. Will the cloud providers find a way to get around security regulations?

Can traditional on-premises vendors figure out how to help organizations build true cloud-like environments on-premises? VMware seems to have taken a step in this direction with Project Pacific, and making Kubernetes the control plane for vSphere. They also have a data-center-as-a-service offering with Dell EMC in VMware Cloud on Dell EMC. But will orgs grasp what these offerings mean and buy into it?

And we haven’t even touched on edge and what that could mean in the scope of digital transformation.  If you think of all the data centers that were evacuated, could they turn into edge locations for cloud? What if you build out an infrastructure that devs can drop their environments onto to take advantage of low latency to your data, or to extend their cloud infrastructure into areas traditionally underserved by the public cloud?

Maybe most importantly, will cloud vendors and traditional infrastructure vendors stop the arms war and start working together?

The future is now

We are in the middle of a digital transformation that will reset computing probably for the remainder of my career. What are you experiencing? I’d love to hear about it! If you’ll be in at re:invent ping me on twitter [ @gminks ], let’s chat! Otherwise respond in the comments, I’d love to hear if I’m way off-base.

I attended Edge Congress last week in Austin, and the conversations were focused on computing on the edge. This post outlines what I learned at the event. In full transparency, I was given a press pass. However, no one has reviewed or approved any of my posts about the event.

Edge is an important part of digital transformation, and traditional ops people should be paying attention. This is one of the areas that is going to change computing, and you don’t want to be left in the land of legacy.

Computing on the Edge – you already have the tools

The host of the event encouraged everyone to think about three elements of edge:

• For Developers: what networks does your app need? Have you made sure the app is able to transverse all the networks that you (or your application owner) expect?
• For Developers/Application Owners: Define Latency. What does real time really mean to you?
• For Developers/Application Owners: Define the importance of your application – what happens if something goes wrong?

For operations folks – doesn’t this sound familiar? This is where we try to get with current datacenters. Container-based applications, especially if deployed to on-premises and Edge locations, will make this exercise even more important and definitely more complicated. Data center hygiene will just become that much more important to the Edge.

Don’t rinse and repeat at the Edge

The first speaker was Joe Reele of Schneider Electric. His advice: we have to change the way we’re designing and deploying for the edge to reap the benefits. Said another way, we can’t look at Edge as an extension of the datacenter, and we have to start looking at Edge with the same economies that we have learned to apply to the cloud.

He explained the paradigm shift from old infrastructure deployments to self-reporting hardware that is managed as a complete microsite. His description sounded so much like VMware Cloud on Dell EMC that I had to ping Kit Colbert.

Hey @kitcolbert thinking about you today @ #edgecongress pic.twitter.com/SZSTRpC1MS

— word girl (@gminks) November 6, 2019

He also encouraged us – operations – to build out the capability on the Edge Don’t look for use cases, they haven’t manifested yet, and won’t until developers and application owners understand what’s being built out for them.

We need to build Edge as a total package that doesn’t increase risk to the application owners if we want to reap the benefits of innovation that we’re being promised. If we build it out like the same old edge and remote locations, there’s really no compelling reason to use it.

Joe Reele: Edge needs to be looked at as a total package that doesnt increase risk. #edgecongress pic.twitter.com/4HsZ8nMkmk

— word girl (@gminks) November 6, 2019

Where *is* the Edge?

This question came up several times. To many giggles and laughs. Of course, this is a question operations will ask, we have to build it and nail it to the ground after all. But that question was compared to the “where is cloud” question of a decade ago.

Edge can be where ever applications need the benefits of low latency, or bandwidth. There was even talk of re-purposing older datacenters to become Edge locations.

Don’t fixate on the physical where. Don’t get trapped into thinking this is the same as CO-LO or ROBO. Think bigger, think differently.

Edge use case: Fix gaming lag

One of the great use cases was from Mathieu Duperre, founder of Edgegap. Edgegap is a B2B company that helps gaming studios solve one of the most dreaded problems in gaming – lag.

If you play on online game, lag is when the latency between you and the online platform is so bad that you tell the game to do something, but by the time it gets to the game server it’s too late, you lost!

As Mathieu explained, gaming is an almost $150M industry.

So beyond subscriptions, gaming has competitions for huge prize money. The last thing a gaming studio wants is for a famous gamer to play their game and experience lag while they are live streaming.

Edgegap uses all sorts of telemetry to make sure everyone playing the game – no matter what device they’re using to play, and no matter where the players in the game are located – have a kickass game.

One example Mathieu gave was using telemetry to recognize that at 2PM around a certain location, load on game servers would spike. This could be because all the kids were getting out of school and logging on. With this info, Edgegap can deploy game instances to a location closer to the kids, and they all could play lag-free.

Computing on the Edge – what Ops needs to know

Computing on the Edge is going to get a lot of marketing attention. But what is the reality for operations folks? Edge is a huge opportunity for us – this is our sweet spot. You’ll need to learn how to talk to your development teams, and to application owners (you should already be doing this).

If you understand what these teams expect, and what they want to build, you can build the environment that they need. Start there, and then demand that your vendors help you achieve that goal.

We’re going into a new world, but it really isn’t uncharted territory. We’ve been here before, but don’t get pushed in a corner. It’s our time to shine!