Mid-Year Review: Swearing Chatbots, CrowdStrike, and Automated Mediocrity

Kyle Nelson: What we’ve seen in the market is that organizations are trying to figure out how they reach their customers in a more innovative way and what has potentially been pushed aside is longer term reference architectures. Cloud spend being out of control, legacy infrastructure just limping along as much as people can to make sure that they’re not having to expend capital expenditures and keep their financials pretty flat. So, I think it’s a direct correlation to the feedback from an industry perspective. We’re hearing it on a daily basis that organizations are wanting to optimize their spend in the cloud. Organizations are wanting to understand how they can take legacy tech debt and squeeze out some of the debt that exists and then also innovate on top of that. I mean, AI clearly is a needle mover in terms of technology and how organizations can leverage it as part of their broader IT strategy or even from a business perspective. So, I think it’s a validation point that organizations are trying to figure out based on this current financial climate, how do we drive cost out of our organization. And still innovate using some of the latest and greatest new technology.

Jim VanderMey: I completely agree, Kyle, and I think that what we’re seeing to summarize what you just said is they are trying to optimize the back end so they can innovate on the front end. And that and so driving out costs while still maintaining the reliability, security, and performance architectures that people are trying to achieve. In order to meet the needs to change the business on the front end is the fundamental tension that we’re seeing.

Kyle: Yeah, for sure. And I think the interesting part is they’re all trying to do it with fewer resources. Nine out of ten customers we’ve talked to that have done some level of migration to the cloud have not effectively driven out cost optimization or even technical optimization, meaning they may have just lifted and shifted something out of the data center into the cloud, but it wasn’t optimized in any way. So, I agree with you. I think that’s something that the industry is dealing with today is a lot of people tried to figure out how do I get stuff out of my data center and into the cloud and instead of lift and shift, it should be lift shift and optimize from a strategy perspective.

Jim: Yes. Yeah, they didn’t adapt to a new way of acting. They simply pushed their infrastructure to another space. And that’s part of what the movement to the cloud enables is different approaches to infrastructure deployment. It also creates new risks and I think that the way that those risks are managed is different in the cloud than it is on Prem. And again those, those shifts of behavior have to be aligned.

Kyle: Sure. Yeah, agreed.

Danielle: So you’ve seen huge increase in digital spend early in the pandemic, people were moving quickly innovating because consumer demands changed on a dime. At the same time, maybe the infrastructure was either ignored or they just didn’t have time to think about it. And also they didn’t have maybe the motions or habits as you’re thinking about the cloud, so is that part of the reason then why people are now starting to focus on those things?

Jim: Absolutely. We all know what year it was. It was 2020. And so in 2024, we’re at the tail end of the three to four year capital cycle that was related to the spend that was related to COVID investment. So, whether it’s the digital products that were built during that time, it was the collaboration infrastructures that were leveraged at that time. It was the cloud infrastructure, or it was the data center investments. We’re now at a place where organizations are looking at that investment and saying, “Am I getting the return out of that?” They’re looking at other capital investments looking at their facilities, you know, are the facilities matched to the hybrid infrastructure, the hybrid workforces of today? Likewise, people are asking are our technical and IT assets aligned to the hybrid workloads of today?

Kyle: Yeah. No, I think that Jim’s absolutely right. And I think the double whammy to that is the current economic climate that we’re currently in, right? So, we exit COVID with those investments being made and then, when it’s time for a refresh, organizations are based on where we are economically with inflation and interest rates being extremely high. The ability to borrow against the credit line to make investments that those things are all in question. So, the easiest way for our customers to be able to afford to do these things is to work with organizations like us to optimize what they’re currently doing and help them plan out what the future looks like. So, I  think it’s a double whammy, right? We’ve pivoted hard in certain areas, COVID, post COVID, and then land in a specific spot of economic uncertainty. That’s also driving some behaviors as well. Good and bad at the end of the day.

Danielle: Cloud had a huge promise of being more economically valuable. Or maybe you could scale better versus traditional infrastructure, and we haven’t necessarily seen that to be true. So, even though we’re predicting or it was predicted at the beginning of the year for double digit spending cloud, what do you recommend as the initiatives that would really help to move the dial for them either in cost optimization or efficiencies when it comes to cloud?

Jim: So, when you say a term like “move the dial”, that implies that there’s a match between the demand and the supply. So, that’s the first thing is that have resources been overprovisioned? And cloud unlike traditional IT infrastructure requires care and feeding. Because there are sources of change, from service deprecation, from new services that are brought on board, to pricing changes, that have to be managed as active attributes of your IT infrastructure. So, if you keep a workload in a high cost infrastructure that could be moved to a lower cost infrastructure, you’re paying more than you should. If you’re keeping too much storage available, if you’re keeping the wrong type of compute infrastructures. These are all decisions that have to be made, and so when we talked about keeping the behaviors of old style IT, the behavior of old style IT was ­— I put something in the data center and it sat there for three years and then I refreshed it. I didn’t have to pay attention to it during that capital period that capital refresh period. In the cloud, I have to pay constant attention to it. And if I don’t pay constant attention to it, I will not realize the benefits that are in performance, cost, reliability or security.

Kyle: Yeah, I agree with that. And I think Jim’s absolutely correct. I mean, we have customers that are have very seasonal business, right. They have to be able to take a ton of orders over the holidays as an example. In a traditional IT infrastructure, they had to overprovision specifically for the seasonality, whereas in the cloud it’s very elastic, right? They can scale up and scale down as needed and that also helps them manage costs. So, if they can understand that, you know in the part of the year that’s not their peak season, they can run at a pretty low performance. Performance is a key, but they can run at a pretty manageable level, but they have to have the ability to scale up quickly and burst into the advantages that they need, and that’s what cloud brings them. But to Jim’s point, if organizations try to treat the cloud just like they did their normal IT infrastructure in a data center or Colocation facility, your costs can get completely out of control pretty quickly. And I think too how we go about addressing the market, cloud cost optimization is just that, not only looking at how do we help you optimize what you have today but being able to help plan out those bursts, whether you have seasonal business or you’ve got a new product being launched or something to that effect. That’s the beauty of cloud, in my opinion, is the ability to scale up and scale down without having to invest millions of dollars in having hardware that just sits there waiting for that burst.

Jim: And there’s an old term that was around when cloud was first rolling out and that was elasticity, the ability to stretch and contract as demand was needed. And those examples like auto scaling one of our great case studies with Traeger Grills, is an example of auto scaling because guess what? At Thanksgiving, Father’s Day, on the 4th of July, people use their smokers a lot more. That ability to auto scale and then go back down again on the Monday after Thanksgiving, because people aren’t doing as much work on their grill. Epic workloads are when we put disaster recovery workloads for hospitals in the cloud, that ability to run at a very small level through normal maintenance activities and then burst it up when we have to do a disaster recovery test or we have an actual event that occurs.

Kyle: And I also think if you go back to the industry reports that that were mentioned at the beginning, what Jim’s talking about is also part of the value behind the automation right from a cloud perspective, the auto scaling is a perfect example, right? That’s not human intervention saying we have to scale up or we’re going to buy more storage or we’re going to buy more compute in our data center and our people are going to install that and rack and stack it. But at the end of the day, I think that’s the other value-add from the cloud perspective is you can build automation into your environment and allow the cloud to expand and contract accordingly. So it goes back to efficiency of your staff and clearly at the end of the day, you can wrap dollars and cents around that because I think that’s it. The automation of it processes from an infrastructure perspective is super impactful, but on the value scale.

Jim: We’ve focused a lot on the conversation around cost and the value that automation brings from a cost standpoint. But for me, the other piece that automation allows for is increasing the reliability and predictability of change — I can make more change faster with reliability because one of the  ways that IT traditionally tried to improve reliability of their infrastructures was to reduce change and make change more costly or more predictable. And we had a whole range of disciplines that around controlling change. Now, the inability to change to meet the needs of the business is a fundamental barrier to the business adopting new technology or growing and changing. And so, automation also allows us to introduce change with reliability. If I manage for change with reliability and cost at the same time. I can in my mind, I can more effectively meet the needs of business in 2024.

Kyle: And the predictability for our customers’ customers, right, if they feel like they can service the customers better because automation is built into their organization, it just makes the experience much better from their customers’ perspective.

Danielle: So we’ve talked about cloud, what other investments have we been seeing?

Jim: You got to talk AI.

Danielle: It’s been painted as both a villain and a hero, right? So, what is it really?

Jim: It’s both. It really is both.

Kyle: It depends on how you look at it and what you your organization needs from AI, right? Is AI being leveraged to offset cost from a human capital perspective? OK, that could be seen as both the villain and the hero. Customers that we talked to are at different phases of maturity, right? Some are scared to death of AI. And what’s that going to mean for their business? But they’re curious because they know that they should be able to leverage technology to benefit them as an organization, all the way to organizations that we talked to are very mature when it comes to that. They’ve invested a significant amount of money in AI and to be honest, the common denominator is organizations that have a solid data strategy traditionally are going to be further along on the AI side because it all goes hand in hand. We’re seeing customers from the entire across the spectrum in terms of where they are in the AI journey. So I think it’s dependent upon how you look at it. It could be seen as a villain or a hero, but ultimately, it comes down to as an organization, as a customer of ours, how do you want to leverage AI in your organization to help you move the ball forward and that’s such a fascinating conversation for us to have. And back to the, you know, strategic nature of what we do, that’s what we do for customers is help them to figure out how AI can help enable their business to grow through the changing economic condition and customer demand, so on and so forth.

Jim: Because our Vervint brand promise is around strategy, experience, and technology. That experience piece is so important to how we approach this because we’re thinking about employee experiences or we’re thinking about customer experiences that are impacted. And I like to say that we design technology to fit into human systems as opposed to putting technology systems in place that humans have to fit into because imposition technology is hard on people. In today’s employee climate and customer climate, if you make something hard on people, they’ll leave, they’ll find someone else to do business with, they’ll find some other place to work. And, so, how do we think about the experiences of people who are using these tools and do so in a way that is  beneficial as well as creating new value and cost effectiveness?

Danielle: You mentioned the data piece of that, Kyle. I’m not sure you know when you start to think about AI, people don’t necessarily think of data in AI going hand in hand. Maybe talk more about what that means and really then what is the actual starting point for AI for a lot of people.

Kyle: Yeah, this is probably Jim’s specialty is probably in this area. I can just tell you from the customer’s perspective. At a high level, you have to be able to feed the LLM’s, you’ve got to be able to feed the AI engine with data to get any answers out of it. So. If you have not properly architected or built a  strong data strategy, it’s really hard to point AI towards just random data because you it would be hard to predict what the outcome is. If you have a solid data strategy, it’s much easier to train LLM’s to come up with the outputs that are necessary. So yeah, I think they have to go hand in hand, right? It’s really important.

Jim: One of my favorite recent stories was an LLM that was pointing a chat bot that one of our clients had an interaction with. And the chat bot was using customer interaction data as its source for developing the conversational LLM and people started swearing at it. Because they were frustrated with the chat bot’s response, which then created training data for the chat bot. So, the chat bot then began swearing at the customers. So, when it got to the end of the query and reached the end of its ability to manage data, instead of transferring it to a human, it started swearing at the customers. And, that’s the kind of thing that that LLM’s will do is we like to say that they’re truthy without being truthful. And so you have to be cognizant that these systems are not like deterministic coding, where if you write a piece of code and it executes the same way every single time. Rather these are systems that are going to evolve with the data that is being fed into it. So, data quality, data sourcing, data privacy, data security. These are all parts of the data governance and data strategy process. But it’s also that you’re limited based upon the quality and amount of dat. One of my favorite examples is that there was a research space that John Hopkins University was working in and they said we don’t have enough data to build a good artificial intelligence model for a particular space because their data was represented by a population of people along the East Coast that were there patients and they needed to have a more diverse data set in order to be able to have a good model developed. So, I think that understanding what you’re trying to do with the AI so that you can align your data strategy to that and make sure you’re not creating promises to the to the market, to the employees, to your stakeholders that can’t be supported by the data that you have access to.

Kyle: Yeah, I think I think that that part is the key. Also, the AI in general  is changing so rapidly. How can we help organizations stay ahead of the curve? I think that’s the value-add that we bring because as a as a customer organization, you may have data scientists on staff. You may have other AI trained people, but you’ll only see the world from your lens. So, it’s really good for our customers to have another voice at the table. To help continue to evolve their AI strategy over time just based on how quickly the technology is changing.

Jim: And to tie this back to an earlier point, think about the run state. We’ve got some really great work that our team has done around the concept of LM Ops, machine learning operations. And how do we create operational structures that support the development of these models? We’re taking philosophically the approach Jake Trippel, a member of our team has said we’re going to be evolving into a world where we have many small models in place that people are going to deploy small models to get incremental benefit in a large variety of spaces in their organizations. The ability to care and feed for those many small models is going to be a new imperative and skill set for IT. So we’re setting up Vervint’s offerings in the space to support the ideation around a small model development, to support the build of that small model, the data strategy that underlies that small model development, and then the care and feeding of those models. So that they can be sustained.

Danielle: So, kind of like cloud AI is going to require some behavioral changes. It has to be fed differently than maybe what we’re used to. It’s also not actually new. I mean you, Kyle, you’re like people have PHD’s and AI. Talk about what the evolution of AI has been like and what other AI things we may know of. Because I think when most people think AI, they think Chat GTP.

Jim: Well, that’s the large language model that has captured everyone’s attention, but we’ve been using deterministic AI for years. And it’s machine learning. We’ve got traditional statistical modeling that has been used and so there’s a wide variety of learning from data and applying data to problem spaces that has been overshadowed by the Generative AI, and I think that’s the that’s the important point is it is Generative AI. It predicts the next thing based upon a sequence of data that it’s been supplied, whether visually generative data or it’s speech, or it’s text. It is generating new content from content that has existed before, and that’s a new application. But there are many other applications of AI and so matching the model of AI to the problem space that we’re trying to solve is very important, because if you apply generative AI to the wrong problem, it’s going to be bad.

Kyle: And Jim’s right, I mean, if you think back to the handshake between data and AI that’s been going on for quite some time, I mean the technical terms around data warehouse, Data Lake, Data Lake House, those things have been going on for years. I actually had a customer previously that was a commodity trading organization and in their data lake they wanted to layer in historical trading data. Bloomberg data of current yields and crops, things like that. And then even National Weather Service data into the data lake. But they would just run queries against that data to try to do better. Predictive analytics in terms of trades. Now with AI, to Jim’s point, that’s more automated, right? So it all goes back and starts with the data. Where do you get the data? What are you doing with the data? Because without that you can’t go automate the predictability of what those AI models might look like. So, you know, we’ve been seeing this on the ground for several years. But to Jim’s point, chat GPT and others,  Copilot, are at the forefront and some of that is marketing spin. But at the end of the day it is evolving for sure. But the foundation was laid years ago in regards to this technology.

Jim: One of the phrases that I like is that your job will not be taken by AI, but it could be taken by someone who uses AI more effectively than you do.  IT systems have supported transactional environments. Historically, financial trading environments. That transactions were managed and then you had robotics and the automation of work differently. And so you see factories that are using a radically smaller number of workers to make the same products, and so the productivity gains. So we have productivity gains for transactional systems. Productivity gains for manufacturing. And now we’re seeing with the generative AI tools productivity gains for the creative spaces, for writing, for content development, for creation of ideas. And so, we’re seeing the automation of the traditional white collar workforce, management level workforce, and the tools that they use for their work, so people that are able to leverage those tools more effectively will have a competitive advantage in the marketplace versus those that do not.

Danielle: Jim, I’ve heard you talk about how AI can automate mediocrity and how it’s obviously if it’s tied back to data. And the quality of that data, we have to be careful of AI then, right? Like we cannot just assume that what it’s giving us is true, it’s not Google.

Jim: The phrase automated mediocrity is because if you use any type of data source, it’s going to pull you towards the middle of the bell curve. So if you’re a person at the low end of the bell curve who’s trying to do something, it will raise the quality of your work up. But it will never do the most innovative, thoughtful work that we see people do when they are applying their creative best. So what we see then is that people can leverage that for getting to the middle faster and then figure out how to get to the best work after that. And I think there’s just a whole realm of possibilities that are implicit with that, but there’s also risks associated with that.

Kyle: If people build AI models and then they just trust that the data that gets spit out is accurate, it does require human interaction. As an example, I’ve plugged in Copilot to my outlook mailbox and calendar, hoping that AI could help me automate certain things. Well, I still had to go back and manually do certain things that I thought were going to be automated. I get a lot of people reaching out wanting to meet, to try to sell me something and it would automatically put that on my calendar. And I was like, whoa, that’s not what I wanted it to do. So, I think that back to the evolution of AI, it’s just going to get stronger and stronger in terms of how predictable the outcome is. But right now, it it’s very human centered and should continue to be that way.

Danielle: OK, so when we go back to what was predicted at the beginning of the year — cloud, AI, automation. Security was also a big one. We’ve certainly seen some examples of why security is important this.

Jim: Well, and I think any commentary around what does the middle of 2024 look like from an IT standpoint would be difficult to not cite the July 19, 2024, CrowdStrike failure — faulty channel file fed to a widely deployed tool that led to a kernel fault, blue screen of death in tens of thousands of Windows servers. And that really highlights something that is a new risk that the cloud world brings to bear is what we’ve called supply chain failures. The complexity of the environments, the rapid pace of change that we demand from our vendors leads to more opportunities for unreliability to be introduced into our environments, and I think that this is something that that the industry hasn’t fully wrestled with. Yeah, and is worthy of a larger conversation.

Danielle: We’ve got five months left of this year. How would you describe how you think the second half of 2024 will be for the IT industry?

Kyle: I think the one word that comes to my mind for the second-half of the year is evolution. And that’s a very broad word because I think many of our customers continue to evolve their IT strategy based on the technology that’s been developed or being developed. I also think organizations are in this new normal of how they leverage their IT investments to benefit their customers and their own internal systems. So, I just think IT in general is an evolution moving faster than ever, and I really feel like, you know, being able to predict IT patterns out further than 12 months is crazy because I think it changes so fast. So as our customers are thinking about the future, it really is about evolution in my mind.

Jim: I think that it’s both extraordinary novel in the times that we face while still extraordinarily grounded to the disciplines we’ve had in the past. And this is where I sound like an old guy, because often times the problems have different words around them, but they’re the same problems. They’re the same pattern. And so, being able to link those good disciplines like we were talking about building good disaster recovery architectures. With the presumption of failure or doing cost optimization or building automation, those are good patterns from the past. They’re being applied to new spaces. And so it’s a good merger of old and new. And I think that that’s I had another word in mind, but it was completely irrelevant to the conversation. So, I’m not going to say that. Because Telestration didn’t really apply, it was close. It was the first word that popped.

Danielle: On that note, thank you both for being here. Appreciate the time you took to walk us through how you see the rest of this year playing out. Thanks for joining us for this episode of 10,000 feet, the Vervint podcast. To learn more from our thought leaders and the services we provide at Vervint, visit vervint.com.