Catch this episode on YouTube, Apple, Spotify, Amazon, or Google. You can read the show notes here.
What does implementing a Zero Trust strategy actually look like in an organization? Nearly a year into our podcast’s journey covering how practitioners view, define, and apply zero trust, it’s time to look under the hood at how a notable organization put its strategy into motion. This week we chat with Bloomberg’s Head of Information Security Architecture and the Information Security Program, Phil Vachon, about how they transformed their security organization with Zero Trust.
Most interestingly though, while many organizations are just now exploring how they will start their zero trust journey, Bloomberg was ahead of the curve even before covid thrust the concept into the limelight.
“I will always say it is continuing to be a journey. It's not a destination,” said Vachon.
Key Takeaways
Zero Trust Principles
Zero trust is not a new concept but has been repackaged and branded as a solid ideology.
Zero trust involves three principles: trust but verify, assume compromise, and strong posture.
Zero Trust Journey
Zero trust is a continuing journey, not a destination.
Zero trust requires a good mindset about how to implement controls and how to reason about security architecture.
Zero trust is not just about securing the corporate IT estate but also about securing the data center estate and the communications between components.
Challenges in Implementing Zero Trust
Balancing security with usability is a challenge that must be addressed to enable a high-collaboration, low-friction workflow.
Bloomberg leverages many SaaS services for collaboration, but they also have their own core services that are still on-premises. They focus heavily on their offerings on-premises and have a big drink-your-own champagne culture around them.
Editor’s Notes
Writing a bit… robotic? I’m using a combination of Notion’s built-in generative AI and ChatGPT to help write our show notes because things are a bit chaotic time-wise. I’ll return to properly written follow-ups after Drataverse at the end of June.
PS, Bloomberg is so much more than a media organization, and today’s conversation comes through the lens of its business capabilities, hardware, and other services.
Weekly InfoSec Headlines and News
Some impactful news stories and community posts from the past couple of weeks.
Meta Fined $1.3 Billion for Violating E.U. Data Privacy Rules
Following UK antitrust order, Meta sells Giphy to Shutterstock for $53M after buying it for $400M
Cybersecurity Chiefs Navigate AI Risks and Potential Rewards
Journey to Adopting Zero Trust
Covid was a turning point for many organizations to rethink their cybersecurity strategy, especially as the shift to remote work was a primary driver. Phil and the Bloomberg team were well ahead of the curve.
They had already been integrating zero trust concepts into their security posture and had been working on securing their external perimeter. However, they realized that they also needed to think about what could happen if someone got inside their perimeter, which led them to focus on having layers of defenses and protection.
Bloomberg's adoption of zero trust was not a one-time implementation but rather a journey. Vachon emphasized that zero trust is not a destination but an ongoing process. He also mentioned that a good mindset about implementing controls and reasoning about security architecture are critical factors in the successful adoption of zero trust.
Challenges in Implementing Zero Trust
Balancing security with usability is one of the significant challenges in implementing zero trust. Bloomberg leverages many SaaS services for collaboration, but they also have their own core services that are still on-premises. They focus heavily on their offerings on-premises and have a big "eat your own dog food" culture around them.
Vachon emphasized that zero trust is not just about securing the corporate IT estate but also about securing the data center estate and the communications between components. He also mentioned that Bloomberg has been integrating zero trust concepts into their security posture for a while now, and they have been working on securing their external perimeter. However, they realized that they also needed to think about what could happen if someone got inside their perimeter, which led them to focus on having layers of defenses and protection.
Adopting zero trust is a continuing journey that requires a good mindset about implementing controls and reasoning about security architecture. It is not just about securing the corporate IT estate but also about securing the data center estate and the communications between components. The challenges of balancing security with usability need to be addressed to enable a high-collaboration, low-friction workflow. Bloomberg's implementation of zero trust is a great example of how a large organization can adopt an effective and efficient security posture.
Episode Transcript
This transcript was automatically created and is undoubtedly filled with typos. As usual, we blame the machines for any errors.
Elliot: Hello everyone. Elliot here with AZT, the producer, alongside my co-host Neil Dennis, who will do most of the talking as per usual. And today we have a wonderful use case about zero trust at one of the grander scales that we'll be able to You know, cover in, you know, I, I don't think we've actually ever touched a proper use case like this.
Most of our zero trust conversations are realistically about people implementing it at an early stage. So, without further ado, I'm going to hand this off to you, Mr. Phil Vaon of Bloomberg, who will discuss your particular role. And then we'll just kind of go down that rabbit hole of, you know, what Zero trust looks like over at your organization and maybe some of the journey that took you to get to where you are today.
Phil Vachon: Oh, well, well, very happy to be on, on the show today and, and really great to be chatting with you both. So my role at Bloomberg I'm the head of information security architecture and the information security program. And, and really what that means is, is my team and and my organization are really responsible for the forward looking vision of how Bloomberg secures its technology and its infrastructure.
So we look at you know, what do we need to build and what are the technology we need to build in house as well as we look at adoption and how we bring in other technologies that can support those, those goals. Zero trust of course, being a part of our strategy overall for how we position and posture ourselves.
Elliot: Very cool. So obviously the concept of Zero Trust has been around for about a decade. So it's not necessarily new. It's also maybe a repackaging. And maybe we can just kind of, silo this a little bit or narrow it down in your mind, what does zero trusts mean, at least as it applies to your business?
Phil Vachon: great question because one of my challenges always is that, you know, everyone is doing zero trust and every vendor. He's a zero trust vendor, at least until log for japp. Then everyone became a software supply chain security vendor. But that's a different story. So for me, zero Trust, actually Microsoft embodied it best.
In, in, there are three zero trust principles, which is essentially you know, boiling it down, trust, but verify, you know, you're continually checking the identity and authenticating any participant in it. At any level, you're assuming compromise and you're assuming that any of the. You know, devices or users or whatever might be compromised when they're interacting with the system.
And then you are making strong assumptions about posture. What is the posture of a device? And, you know, ensuring that you enforce that policy consistently. Bringing yourself to a, a least trust posture so you don't have, you know, more privilege than you need to do your job or to accomplish whatever it is you need to accomplish.
Elliot: Excellent. Yeah, I fully appreciate that perspective. There's obviously different takes where we can look at CISA and NIST and look at the different architecture and it gets a little more in the weeds and technical, and I'm sure with your team they're a little more well-versed, but at the business level, having those three different pillars makes perfect sense to, you know, go for that.
As far as the definition,
Phil Vachon: And it's funny cuz if you really want to geek out about it, like if you go back to like Saltzer and Schroder, which is I. Believe was published in like the mid 1970s, that paper. You realize that a lot of those principles that we're talking about, they said it in more words. I think they had seven principles.
But in the end, you know, none of this is actually really that new. It is exactly what we said. It's the packaging of it that we're talking more about and the techniques that we're now able to accomplish because of how device posture has changed and how various technologies have evolved around transport and that sort of thing.
Elliot: Absolutely. So I'm curious.
Neal: Sorry.
Elliot: it to deal, to give the good summaries. No, that's, that's, that's, I mean, you're, you're just well aligned. We'll put it that way. Usually when we chat with folks that is absolutely within the realm of kind of how we. We sum it up, but occasionally we're looping in a couple vendors and obviously they have their very own different flavor and I'm sure you've run into that.
And I'm sure people also are in your inbox pretty constantly of the new zero trust stamp box. But yeah. So. Before, I guess I hand this off to Neil and you'll go down that lovely rabbit hole that we find ourselves in. I guess the primary question is, you know, what did the journey look like for adopting Zero Trust?
Was there any trigger point? Was it just a, you know, for a lot of organizations, maybe it was Covid and there was like distributed teams all of a sudden, but yeah. Was there anything in particular that pushed you towards it or, you know, were you just kind of getting ahead of the curve?
Phil Vachon: what's interesting is when Covid hit, we were well on the curve of adopting zero trust. And, and, and actually if you look at the overall principles of, of zero trust and, and kind of like the, the way that influences your design, your security, the way you think about security architecture we'd been integrating these concepts for, for many years.
We, we kind of, you know, like, like most corporations, we follow the traditional path. We, we have our own data center presence. We need to make sure that that presence is secure. Obviously, the vast majority of compromise and attacks comes from outside of your infrastructure, so that's already a scary thing to to, to think about.
And so, you know, years ago we started on this journey of making sure we had that well secured external perimeter. But what we also realized is that even within that well secured perimeter, we need to think about what are the things someone could do if they get inside of that. I mean, let's, let's be honest, misconfigurations could happen, or, you know, all the, the types of errors that you want to avoid.
So you need to have those layers of defenses, those layers of protection that we can then use to, to, you know, have some assurance in the event that something does happen that, you know, a, an a malicious party or, or, Malicious insiders can be blocked and not able to accomplish much more than, than there, there should be allowed to, to do their job or whatever it is. So that approach has kind of been at the heart of how we've looked at our security posture and as we've continued to evolve, one philosophy we've adopted is to take those perimeter security considerations and bring them closer and closer to the actual systems and services and users that are. A part of our ecosystem and, you know, the people who actually do the real work that we have to, that, you know, that keeps the lights on the business.
Now one thing, I just wanted to step back from that, cuz, you know, that describes kind of a zero trust journey. But one of the things I I, I like is that you describe it as a journey and one of the things I will always say is it is continuing to be a journey. It's not a destination. This, this whole philosophy.
And, and really it's all about having a, a. Good mindset about how you implement controls, how you reason about how you build your technology and technology stack. And I'll get into that in a little bit of a moment. But also it's about making it clear what is acceptable posture and what is an acceptable posture so people aren't living in a gray zone with you know, a particular application deployment or whatever it may be. So that is how I would at a high level describe it when. We get into the weeds. Now what's gets interesting is there's many different aspects of an IT estate, especially for a technology company like we are. And you know, one aspect of course you think about is the user facing components, the hybrid work environment, our corporate it estate, all of that.
You need to focus on, you know, how do you enable users to do their job, do that job securely ensure that your, your SOC can. Monitor and identify if there's any bad behavior or risky risks or threats that you have to deal with. But then also you know, making sure that the, the default posture of any device that they're using takes into consideration all of the controls that you would put in place, you know, around your corporate network in general, the perimeter of your corporate network if you're using proxy to monitor.
Access to the internet. Hopefully most people aren't freely allowing their employees to access the internet. You know, you end up needing to implement those similar technologies or similar approaches on the device, on the endpoint itself. So the, so that's one angle a lot of people focus on, and I think since like Google's Beyond Corp pay white papers and all that, a lot of people think about, you know, zero trust in the context of of the, the, the corporate IT estate and the the enterprise IT estate.
But really what is exciting to us and what's more important to us is the posture of our data centers, right? We have a vast number of services, tens of thousands of services, microservices. In fact, we were calling them microservices forms trendy to talk about microservice meshes. And you know, that means that we have to craft policies for each of those microservices because we need to understand what services should be talking to which, and we need to make sure that.
Things are segmented based on what their, their role is within our greater ecosystem. And so really the interesting part of the Zero trust journey begins when you start looking at the data center and how do you secure the communications between the components that make up our broader service offerings. And, and that's really where we focus a lot of our energies. Like we have the corporate IT estate, we have a significant evolution that we've been working towards in that direction. But then of course there's the, just that, that, that data center estate and, and how that has had to evolve in order to. Kind of deliver on that vision.
Neal: No, that's kind of fun stuff. I mean, that that's. It's a very awesome, kind of high level explanation of the path forwards, I think. So I have a very big curiosity question. So you talk about, you know, the, the micro segmentation, you talk about asset controls and all those other things which are, you know, fairly commonplace constructs, what you need to, right.
For just any security policy for y'all's infrastructure and awareness. You also brought up, Outside things like cloud-based services like Google. Just out of curiosity, you know, when you start taking into account or have y'all really had to take into account cloud-based solutions as a as, not as a tertiary thing, but as a primary focal point on how y'all implement those security posture.
Phil Vachon: It's a, it is a good question. So we we're kind of a hybrid in that we like I say, we, we run our own infrastructure. We have our own data centers. But we want to make sure we accelerate our time to market, be it for something we're offering to our customers, but especially focusing on our, our enterprise IT estate.
And in that case we do leverage cloud services. You can't beat Zoom, for example, for collaboration. You can't beat G-Suite for how easy it is to group out a document and, and you know, be your library of documents. But yes, with that comes all of those security concerns of. How do you enforce that?
People should only access your G-Suite instance from a, a device that is actually a part of your part of your managed fleet. You know, all those types of controls, what your CASB does when, when someone isn't you know, permitted to, to access access that, that particular SAS product, whatever it may be. So there's, we have a lot of interesting challenges around that because we we obviously. You know, want to enable a very high collaboration, low friction workflow. We want people to be able to work you know, or when they need to, where they need to in order to get the job done. So usability is a big focus for us, but we do have to balance that overall with making the right decisions to protect the business, to ensure sensitive data doesn't end up where it shouldn't or whatever it may be.
Maybe, you know, my, my my recent example, Was you know, there was a vendor that was compromised through a, a plex zero day on a I don't even know if it was zero day, but a plex vulnerability that that ended up being used pivoted to, to gate access to sensitive infrastructure. I mean, when you hear something like that, I mean, that's, that's why we focus so hard on the, on the, on the posture of our endpoints and making sure that, you know, you won't be able to, to pivot from a, you know, grandma's home network to.
You know, our corporate estate.
Neal: Yeah, I, I think that that's, to me is a really good. Mentality for what Zero trust is supposed to bring to bear you. You know, once again, it's never a matter of if, it's just a matter of when something bad's going to happen. It's what you've done before it goes bad. That determines just obviously how expansive that is.
You know, you could have another Solar Winds event, but if the boxes with each of their own solar wind packages are isolated the right way. It stays mostly there per instance, right? There's no pivot points that should be easy to get through without additional exploitation. Pass beyond. I think that that to me is the key factor here that helps, I believe people focus on the necessity is, is understanding that that strategy ahead of time, you know, and, and to your earlier points about.
What is old is new. New is old thing. We echo that a lot and you know, zero trust as a construct really isn't nothing new. It's just being repackaged and branded and finally taking root as a solid ideology, which is kind of cool. Yeah, so I mean, the, on the last question about the cloud piece, so you kinda also hit this a little bit, but the supply chain risk management pieces and, and the whole new sbam junk that goes with that, for better or for worse.
So when we talk about like, Google and just email, you know, you're not gonna go get a bill of materials no matter. I don't care if you're, you know, A big company like Bloomberg or or a giant bigger, larger, smaller Apple, whatever, chances of you getting some kind of blatant bill materials just for a Google Suite is very not likely, right?
But if you're going into other software service provider, sas, whatever those may be, you know that there's some potential supply chain risk mitigation aspects that I think tie into the zero trust mentality, right? And conversations that could potentially happen between the two. So have y'all kind of experienced that at a lower echelon SaaS?
Providers or maybe even with Gmail, who knows? Relative to that kind of construct of if y'all see something, we talk about it. If they see something and then we figure out how to lock it down jointly kind of deal.
Phil Vachon: Yeah, I mean the, the story around SBO M and SaaS offerings is, I, I feel like it's an evolving story and I, I actually find it interesting cause I haven't really seen a satisfactory. Story around some of an offering like Gmail. Sure. We we're not a Gmail shop, so maybe it's just not a problem that really crosses our radar for services like that.
But I, I think right now the question that I'm continually asking around around SaaS offerings and when you even talk about SBOs is really an sbo. Sbam is a snapshot of a state and time, and like how rapidly is that snapshot or is the world a reality? Deviating from that snapshot, first and foremost.
But second of all, like when I look at you know, kind of all the integration points in general to me the SaaS offerings tend to be on the better side in terms of, especially from a company like Google, in terms of where the risk is it lies and where the risk is addressed. So, you know, I think it's kind of like you gotta pick your battles a little bit on that sort of thing.
But like I said, we're. We have this interesting hybrid where we, we leverage a lot of SaaS services for, for collaboration and all that, but in the end, you know, we have a lot of our core services that are still on premise for us, that we run ourselves. You know, emails a core offering, actually the Bloomberg terminal product through message.
And, and you know, we, we we definitely use collaboration technologies like Microsoft Teams. But you know, most of our collaboration actually happens through Instant Bloomberg, which is a, a an instant messaging product built into the Bloomberg terminal. So it's just kind of interesting cuz our, our posture around a lot of this is, is maybe a little softer than other companies because we, we just focus so heavily on the, the offerings we have on premises and we, we have a big eat year own dog food culture around around those types of offerings in general.
Neal: No, it's neat. You know, there, there's. When you're at a scale of economics and you can own the process behind the tools itself, it makes a big difference. And obviously where the security implications lie or what mitigation strategies you can have, I think that's kind of fun. And it, it's, it's, it's something I've personally seen echoed at larger corporations in general that, that have had a tech startup or, or internally or have eaten tech startups to do things better.
Most of your larger. Hmm, most of your larger providers. Mm-hmm. I, I also worked on the retail side in the Intel for Retail ISAC and some other you know, our, our big blue in Arkansas, you know, they're very similar in their approach. They have a lot of their own ingrained in-house things. If you go to Walmart and you swipe your credit card on a, on the system, that os on the POS itself is theirs.
They built it, they own it, they control it. And you're not gonna
Phil Vachon: okay if you love it and hate it, so it's
like,
Neal: that's the problem. Right? So there's some, the old adage, security through obscurity, you know, if you, and this is what benefits the OT world so much of even today, is that everybody's got their own special flavors of picking ot.
But if you got your own special flavor of pick a kernel, you know, it may be the most corrupt thing on the planet, but it's gonna take a threat actor a little longer to figure that out in theory. Right? Yeah.
Phil Vachon: there's a, there's a little bit of being able to hide behind obscurity, but I don't know, as a, as a habitual reverse engineer of iot products, they I, I, I think that obscurity is, is narrower and narrower, especially as tools have just gotten so much more sophisticated for those, those analytics, like if someone's really determined, they will do it.
Neal: well now you got things like the little flipper that you can buy for 160 bucks and walk around and do all sorts of fun crap.
Phil Vachon: Grab badges from people on the street or
Neal: I mean, I, I ordered one a month ago and it's on the back order list. Just to be fair.
Phil Vachon: Maybe they flagged your orders as being suspicious or something. Oh, it's a security person. We gotta be careful.
Neal: Yeah, well if they've looked at the stuff I've bought in the past, then yes, the answer's yes. Cuz I have that entire capability in about five different devices
Phil Vachon: Yeah, exactly.
Neal: it in a single anyway, so, you know, that's the fun piece though, right? Is is to your point though, and where security through obscurity only gets you so far and nowadays the, the.
Power curve to being a, I wouldn't say necessarily sophisticated threat actor, but being a capable threat actor at a lower echelon of knowledge with a higher capability technology has only been ramped up exponentially, and it's going to continue, right? We've seen this for the last 25 years. You know, script kitties in Brazil in the early two thousands.
Some stuff in China, Russia, a little bit from a script kitty perspective, but
Phil Vachon: Talk about script kitties in the UK and Brazil last year with lapses.
Neal: Yeah, yeah, yeah. But what they're doing now compared to what they were capable of doing, you know, 25 years ago, if you were a script kid 25 years ago, you, you had skills that were hireable. As a programmer, you may not be great, but you could get hired. Nowadays as a script kitty, so to speak, your skills barely get hired at like a gas station.
Man, that's probably why they're doing what they're doing. slight anecdote to get back on track. The safe flight repairs, safe flight replace, you know, everybody knows that slogan. People would be very befuddled to find out that replacing AUTOGLASS only makes them pennies compared to what they do globally on everything else.
Yeah, their primary stuff is actually managing insurance offices and phone call centers, funny enough, and brokering insurance between things. Right. No, so that's cool. I, I think that's fascinating. You know, we've got this public presence of what we assume is something, and then you find out awesomely enough that that pitten
Phil Vachon: We we're kind of like the best cap tech secret. I think in some respects we. Are a you know, 8,000 plus strong engineering first technology company in the heart of Midtown Manhattan. And we have th, you know, thousands of engineers in the tri-state area alone. And and, and I mean, like, it's kind of been an interesting evolution for us as a company.
You know, we, like Mike originally saw the value of introducing transparencies into the, you know, fixed income markets and that sort of thing. But the product, the vision that he had and the product we needed to build to deliver that the, you know, off the shelf technology to do that didn't exist. We built our own, effectively Bloomberg terminals.
Literally the Bloomberg terminal was a physical device that set on your desk and there was like a bunch of cables that went to a box off in, in a, in a wiring closet somewhere that would, you know, get you the actual data that was needed to display all that information. And, you know, we had to build everything from the, the hardware we had to.
Customize the monitors for this thing. We this is the mid 1980s, right? And built our own keyboard. We still build our own keyboard, actually to this day, believe it or not. You know, you'll see it's multicolored. It's very, very distinct to the device. But what's interesting is like we've always been on this kind of cutting edge, you know, very focused on delivering rich information about financial markets and, and products and instruments.
Us and we've had to innovate a lot to do that. We were often ahead of the curve. We built our own variant of what I would call hypertext almost, you know, in the 1980s to, to, to play pages of information about bonds. We built our own private IP network before the internet was as widespread as, I mean, anywhere near as widespread today.
In fact, we were one of the largest private IP networks in the world until Google and Facebook and, and, and Microsoft eclipsed us. And, and you, you just, you know, you. We are this hidden technology company that's actually had to innovate a lot in these spaces. And, and what's actually really cool, and, and part of why I think, you know, this is relevant to kind of our journey with around Zero Trust and all that is we've built a lot of technologies, you know, in-house we've developed and innovated to, you know, build our own microservice mesh before anyone was talking about things like G RRP C or proto buffs or any of those technology that everyone uses widely today.
And you know, then we've done this great job of adopting. So when what we built was, you know, surpassed by the rest of industry because, you know, like web browsers became a thing and because you know, microservice meshes were standardized, especially around like modern containerized you know, infrastructure orchestration like Kubernetes.
We have evolved to adopt those things as well. And, and actually even supplant our own custom versions where, where industry has exceeded us and then we contribute. So we become part of those, those those particular environments and then the, the open source communities around them. So it, it's kind of a, it been an interesting, it's an interesting journey for us as a company, but from a, the kind of like pulling us back to the zero trust perspective you know, zero trust concepts have been baked in actually to how we deal with, you know, service, service authentication and authorization controls for, for many, many years for us.
But what's been interesting is we've seen, especially through cncf. A number of very interesting technologies crop up like Spiffy Inspire which are service authentication technologies standard out of the box works with just about anything actually, which is kind of cool. But then also looking at you know, authorization tools like OPA and open policy agent and all that.
And what's really, I think, exciting about this is no longer are we having to kind of build our own versions of this no longer we have. To maintain our own versions of this, but we can actually start adopting these technologies, these cloud native versions, technologies in-house. And then as you know, we move closer to our customers in the cloud to deliver services directly to them.
Cuz you know, we have a lot of data that we're delivering to our customers who are very heavily co-located in the cloud now. They want, obviously want us to be as close to them as possible for economic reasons and reliability reasons. But you know, the, the fact that we're switching to this mindset on premise as well as you know, bringing these technologies we have for delivery services to the cloud, we're actually in this really interesting position where we can adopt common mechanisms now across, you know, public cloud and, and our internal infrastructure as well.
So, you know, this, this kind of wave of like, we build, we build, we design, we come up with our own technology to come with our own approaches. And then we adopt and then we work to extend and, and we, we work to grow these tech open source technologies as well as kind of, it's in our dna.
Neal: That's neat that that's not unlike what I had to put up with in the government side, military side. But yeah, I, I think it's, it's a purpose-built. Flow ebb and flow. You know, like you mentioned, you have a problem, you need to solve it. You can't wait for somebody else to build a product, especially earlier on in the tech days.
So you go out and you make it yourself and you make it whatever it needs to be. At some point in time, it not necessarily that it stagnates, it's just that market, like you mentioned, catches up to you and other people come up with new thoughts and I think you. You adopt those ideas, you build into those ideas.
And then there are gonna be splinters where you're going, eh, you're no longer what I thought you were, I'm gonna go back and build it myself. And then rents allow the repeat rents outta the, that, that, that's like government procurement 1 0 1. And, and, and. You know, some companies are really good about staying in that Bailey wick and just sticking to what they build and innovating in those circles.
Some companies go back and forth. Some companies obviously exclusively only use what they can find versus what they make. But I think it's, it's a good flow. I like the flow ebb and flow process personally, cuz it allows you to do things really well for a little bit. Learn more ways to do it. Come back and do it again.
And, and hyper focus on critical ideas for a time being. So that's neat. No, like I said, I had, expanse wise, no idea. I knew there was definitely financial securities type stuff involved and ideologies around money and things like that that go towards that. But that's neat. So from, from these product perspectives, so if we think about it from a product, product creation side of the house when you are, maybe this has changed a little bit now that things maybe are probably more digitized, virtualized from a product.
Peace. But are y'all still transitioning from physical type devices or on-prem for your client-based type structure for what you produce and provide product-wise to more virtualized type things, or is there still a mishmash or requirement for both sides of that and then yeah, we'll go from there. I guess
Phil Vachon: Yeah, yeah, yeah. So, so I mean, like I have alluded to, we, we actually, we have our own data centers. We, we we're in this awkward space where we're not quite a hyperscaler but we're not, you know, a company with one. Data center in one location. So it's a it's kind of interesting to see like there's this di this dichotomy of products and all that, but that's a, that's a different complaint.
So a lot of what we do is, it is our own hardware, but we've been heavily focusing on, well, how do we build a cloud internally? So how do we have the, the agility that a cloud-like offering would have for our internal users? And so when we deliver services internally, even. The vast majority of cases, this is actually delivered on our virtualized infrastructure.
It's, it is a you know, a cloud we've built ourselves from, from scratch. Effectively, we are leveraging a lot of open source technologies, of course, but it, you know, all the management and infrastructure and all that is very specific to, to Bloomberg and how we work. The upshot of that of course is that we're, we're able to move very rapidly in integrating features that are specific to how our workloads have to work.
Of course the, the downside is that we do, you know, we end up having to maintain it all ourselves and, and sometimes, you know, a great open source technology might crop up that we're gonna have to work a little harder to, to integrate it, but I think the benefits far outweigh the, the cost there. So, but yeah, we're, we're mostly on-prem, we're, it's our cloud, I'll call it.
Obviously there's always applications that, that are, are a little bit older, let's call it, but generating a lot of revenue. They might still live on, on, on bare metal on particular machines, but, so, but again, wanna talk about part of the Zero Trust journey. How do you actually integrate those applications that you know, generate a lot of revenue, are very important for the business but then also ensure that they're you know, secure and, and actually not you know, in a, in a position of creating a significant amount of risk for, for your security teams and for the company itself. So it, it poses an interesting challenge on its own.
Neal: Yeah, that was kind of, you kinda led right into my next question indirectly, so that, that, that's fun. So from legacy systems, new age systems, whatever you're building, do y'all have, do you feel like y'all have the appropriate policies that have been agreed on? So as these new things. Gear up, you can identify appropriately within reason.
You know, this is obviously this type of equipment going in this piece or this software, whatever it is, right? So then you have buy-in across leadership. Say, check these boxes at a base to do this, or are y'all still kind of, you know, is it still more of a, let's look at the list of what's coming in and then discuss more aptly kind of deal.
Phil Vachon: So, so we're getting better at doing it I'll call it app priority, right? We have well-defined policies in many cases, and we're, we're getting better at ensuring those policies are integrated into how systems and services are. Point. And in fact, we've been making great leaps and bounds I'd say on that over the last five years.
The exciting, I think part of that is, is that you know, it sometimes these policies, especially when you start talking about the, the enforcement of the policy from a, from a technical perspective, you know, your policy enforcement point and the policy decisions they're actually not obvious.
You know, you might think. You can reason around, well, what, what service needs to talk to, which service and all that. But when you actually dive into it, sometimes those relationships need a little bit of work to tease out. And if you have you know, a legacy system or, or you know, some older system that you're, you're supporting, one of the most important parts to remember is that you need to find a way to identify just who is depending on that.
Cuz sometimes you have hidden dependencies that you have no idea existed. You know, it's just the reality of the. an estate, of an IT estate. What is, I think been an exciting bit of research that we've been doing is how do you analyze everything ranging from the binaries themselves that you're deploying through to the actual usage traffic to identify what are good patterns and what are bad patterns, and then rendering policy intent from that.
So we've been spending a significant amount of work actually researching and. Identify developing techniques to do this. And, and the hope we have with a lot of this is obviously it fits very nicely around the way our service mesh and our services technologies work, but you know, how can we apply this to, you know, more standardized technologies, like how do we use this to Ford policies for Kubernetes?
Or how do we use this to infer. Q bar policies, you know, on top of, also on top of Kubernetes, but focused on the workload itself. So we were spending a lot of work actually in, in doing research and investigating how to approach this. But conversely, the, the, you know, some of the most effective ways to actually accomplish this of always being to to basically put everything in monitoring and alerting mode and, and then, you know, Have someone triage those alerts like nine times outta 10. A human will be able to reason around most of the, those cases. But you know, for some of those more complex cases or where you need to achieve scale, cuz like I said, we have tens of thousands of services and sometimes the dependencies are not obvious. We have a lot of really interesting techniques.
We've been working up in house. To support that
Neal: Yeah. You know, I find if you unplug it, you can figure out who's dependent on it pretty quickly. Yeah.
Phil Vachon: prob problem is that if, that, if, if I'm plugging it annoys a client or something like that, then I have to deal with an annoyed client and the annoyed developer, so I talk to the annoyed client or something like
Neal: Yeah. Yo, I'm right there with you. Yeah, I've been there. Yo, on that note, so I'm gonna maybe digress a little bit in time here for this. So I'm, I'm very curious about the, kind of the start of this journey. You know, we talk about access controls, identity management. Obviously these, I think, are all, all, all our totems for what, you know, banners, what we need to think about and, and.
Key things to consider, but I'm, I'm very curious, you know, kind of alluding here lightly to what what was your kind of buy-in like initially when you brought, like when this idea was really table and, and you. You know, maybe like you mentioned earlier, it's just maybe just part of y'all's intrinsic nature to go along this path where it was at, but was there any kind of buy off or any kind of discussions that you really had to focus on to get both leadership and tech to kind of really move down this more concentrated focus of what Zero Trusts meant, or was it just intrinsic to the flow?
Phil Vachon: So I would say getting buy-in was kind of, it was, it was a multifaceted process. And, and what's, I guess, kind of interesting is, There's some of this stuff is in our dna, just in the way we operate, the way we designed our systems, and we've been doing this, you know, our developed our service mesh, you know, in the, the late nineties, early two thousands.
We've been doing a lot of that since then. What I think really did help drive things home was when we, it, it, I think it finally dawned on folks the impact of one little piece of technology that we had on. Our security posture is a firm and how many other ways we could benefit from it. And this is especially talking about the identity piece of the puzzle.
So since 2004, we've been shipping our own biometric authentication technology, which is called the B Unit. It's integrated into the Bloomberg keyboard, but also it was a portable device. Up until recently we, we switched to a mobile app version of it in the last year or so. It. But the the interesting piece about that is it made folks realize that, you know, for example, when we talked about at the time, especially two factor authentication, it was just a big deal from, you know, just general protections and posture, security, posture of the firm.
Having that story where it's like, well, hold on. This two factor authentication has actually been in our dna. Our customers haven't been able to log into our service unless they add their B unit or their keyboard with their biometric enrolled in front of them. And. I think at that point it actually became a pretty easy sell because the innovation, the security innovation that we have as a company, and the fact that it's kind of been in our DNA to approach it this way made it easy to explain to people like, you know, okay, so we've got really good identity.
What does that mean? Well, if we can authenticate every human, you know, every, every client, every employee, all that pervasively, and they have low friction mechanisms to do this form of an app or, you know, leveraging in, in modern cases. Its secure elements built into laptops or, or mobile phones that actually, you know, means that we can start having a better idea just who is doing every action within our, within, you know, our infrastructure and our, its d.
So it's kind of building on the back of that and continuing that evolution that I was alluding to before where we started and all of us started here, right, of focusing on securing our perimeter and really. Moving those security controls closer and closer to services and systems themselves was an easy sell.
When people saw, well, hold on. We already have this great authentication technology here. Why can't we link all those pieces together so that only the person who should be accessing, you know, Phil VA on's email is Phil Vahan and we can prove it because he's got this biometric. And that kind of starts getting into this, the concepts of, you know, least privilege and you know, continuous authentication, right?
Like you, you really. Can start to, to reason around and rapidly, but hold on Wolf. It's only Phil who has this particular device, and he's the only one who's able to authenticate to his email. We better make sure that that is actually the case, you know, so it it, it was kind of the snowball effect working from a lot of those things.
And and I mean, look, the other side of it is, I think one of the great fortunes I have is our leadership, at this company, Mike included, have a great deal of vision, technical vision. That you know, when you explain something like this to them or you explain the value of like, we have authentication everywhere, this is what we can start doing with that.
They actually listen and they are very interested in you know, how do we actually take advantage of these things? Because for us, like in the end, it's all about trust, right? So we want our clients to trust us and because we want to ensure that that. Our brand is trustworthy. We need to make sure we do all these things to keep our data secure, keep our services secure, make sure that only the client who, who should be logging in as that particular and only the individual, the human who should be logging as that particular user is actually that human.
All those assurances actually work to build up that trust in that brand. So it's kind of a long-winded way of story to tell, but I think it all comes back to like we are so heavily focused on technology innovation and we. We really, I think, had this cool technology that was so far ahead of the curve at the time that we developed it with the, with our biometric authentication.
And then I think just rolling that into every other aspect, like that strong identity concept, it resonated
with folks.
Neal: so
Phil Vachon: It became a blur, sorry.
Neal: No, you're good. I, I was gonna say, so for the listeners, I think that that might be a good strategy for y'all trying to approach this is just reinforce some really good, you know, identity access management processes to start perhaps, you know, upgrade maybe a little bit more beyond just basic MFA two FFA structure, and then, Wait, show 'em the value prop there, and then see if they can get buy-in for more appropriate controls.
I, I think it's kind of a, a fun approach and everything echoes back to who's who and validation of who's doing what. Right? So you need to know who's doing it and why they're doing it, and where they're doing it from at least a little bit to start to understand the persona, whether it's human or digital, and especially now more so with AI type interactions and things like that becoming more reality.
Quote ai It may be able to teach itself a new language, but I still don't consider it alive. But whatever it is what it is. But, you know, regardless there, there's gonna be, there's a human machine symbiosis piece here that that's always gonna be there, and it's gonna be even more prevalent now that people have, once again, lower levels of capability needed to access higher technology capacities to do these things.
And. You know, so I'm sure people are already doing this prolifically writing in their API creds and stuff into chat, G P T, doing all that stuff. But how did you know? Thankfully there's tools trying to fingerprint that stuff for us. But yeah, no, that, thank you. That's kind of a fun part of the journey and like mentioned.
You know, you start somewhere and like with any good program, you gotta figure out where that lowest echelon of buy-in possible is so you can start creeping up the ladder to get more. And I'm, I'm a big fan of finding ways to internally make it more secure and robust for who's logging into what, while you still figure out how to start doing the segmentation rules as it
Phil Vachon: Yeah, and it, that, that's the hardest part, I'll be honest. Like it, it. It's funny you know, going, looking back and I'm, you know, looking back to the nineties where like PKI was like all the hotness, right? And we were talking about pki this, PKI that web PKI was a new thing. We weren't calling it web PKI then.
And of course, you know, the government with with CAC and, and all the capabilities that came with that. And and in the end it was, it, what always was interesting to me is that, The hard part of PKI is not the te, I mean, the technology's inscrutable just because the standards are, are very dense and complex, but it's actually the business logic and it's the fact that you're representing organizational structure, you're representing business interactions, you're representing roles and responsibilities of of humans as a part of like what that, that credential, that, that, you know, the cert you might be presenting implies and, and this problem still is fundamental within zero trust.
And, and when implementing a zero trust strategy, it's like that authorization piece is so hard, and I think that's probably where we've invested the most effort and where I think we will continue to invest the most effort because in the end as well, it can't, it can't be high friction, right? If you, if you are preventing business from happening especially if you do it pervasively you know, you, you will find shadow.
It grows very rapidly. And the last thing you want to do is, is have a control being the reason. And
Neal: Yeah
Phil Vachon: how do it crops up?
Neal: so I think I, I like to ask this question a lot but, you know, ease of use, friction and, you know, all that fun stuff for access controls and what that means for these wonderful new. Passwordless environments that people are trying to launch. Right. I, I will say personally, I do agree that the future is not passwords.
I do agree there's other ways that are more, more synergistic with the humans as well as whatever else to be able to get those authentications in play and, and one zero trust echoes a lot of these requirements. You keep, you know, you keep talking about location accesses as, as a good one. I think that's an easy one to really think about.
But what, what's your hot take on. On a passwordless environment for maybe not necessarily everyone per se, but as a basis for maybe customer interaction or low level employee interactions perhaps, or more.
Phil Vachon: like especially talking about past keys, cause that's the, the recent standard that everyone's been been talking about, which is built on top of Fido, U two F or Web often, I guess is, is what it became as, as W three C really adopted it. Look, I, I, I think anything is better than passwords, right?
Because managing passwords is just, it's a, it's a nightmare. It's a risk, it's a, you know, very hard to tell. Like, who, I mean, it's, it's a bearer token, right? It's like, I, I am, I am Phil. Why? Oh, because I presented this password. Well, are you really, Phil? How do we know? Being able to bind to some sort of authenticator, which is what a lot of these standards allow you to do, is an exciting opportunity.
I find usability is still the challenge we're going to continue to face. I don't know how I'd explain it to like a grandparent or explain it to you know, my, my mother's a primary school teacher explaining her some, well actually she's, she's more savvy than others. Cause I think I, I steeped her in it through through my entire childhood and life and all that.
But nonetheless explaining like how Ubi Key works and how, how, you know, important that might be to her being able to get into her email. I, I, I don't think that's a very great experience. And, and so I, I think there's a lot of usability questions that have to be answered. I suspect there will be trade offs that we have to continue to, to accept around something like that in the current incarnations.
So I, I, I think it's one of those things that, you know, early adopters let's, you know, shake out some of the challenges, the usability issues and all that. But I mean, like some of those workflows. Especially when you get to the edge cases, like, I've forgotten my password, so I can't reset my access.
And I also, or, or my UBI key got run over and I don't remember my password or whatever it may be. You know, it, it's just those are, those are really hard to solve. And, and speaking from having done it for a product that you know, is widely used within a, an industry that's very fast moving and fast paced those user experience.
Challenges become, you know, really high friction points for, for users, for, for your customers. So, you know, in, in the upshot is at least I can have a high-touch interaction with the customer. And I, and you know, it's part of our DNA is to make sure the customer is the best possible experience when they're using our product.
And if they hit an edge case, we make sure that they can get past it. We have ways to do it tool through this so forth. But, you know, if you're looking at something like a Google where you have billions of users, or Facebook where you have billions of users, can you imagine like, The number of people that are gonna hit those edge cases on a daily basis and suddenly be locked outta their accounts or whatever it may be.
So, so I think there's, there's just a lot of these challenges. I think passkey is a step in the right direction. But there's just a lot to be solved, I think, from how users see these technologies and use them before we'll see wider scale adoption. And, and I, I kind of, you know, I say this as someone who's been.
The moment two factor off became available for any services. Like, oh, gotta get my YubiKey out, plug that in. Gotta get my you know, authenticator app out, make sure I got those tokens. So I've got the my, my backup tokens, you know, stored in, in my my password vaults you know, secure messages area.
Like I, I'm, I'm a big adopter of that stuff, but like, knowing the time, sometimes I, the mental gymnastics I have to go through to, to unwind some situations to, I run into. I mean, I just, I feel bad for the user.
Elliot: I kind of just want to carry that forward because, you know, I, I love the philosophical aspects that come out of the cybersecurity world and. Usually the consumer side is completely neglected it, you know. don't see any of it. They don't really have any interfaces, interaction points.
So this is not necessarily a zero trust concept or a question around that, or even just in general. But you know, do you feel there should be more Otis and expectation on software creators to take some of that pain outta the process? Like should there have to be. People in every organizations that just have committees just to be able to educate, you know, their end user.
Like, you know, they're not gonna expect accounting to fully understand zero trust, but they should be able to have the benefit of it. So I'm just curious, like what's your perspective as someone who clearly lives and breathe in this world and understands the technical space and has to communicate it?
Phil Vachon: I mean, the best technology's indistinguishable for magic, right? So from, for in that accounting department, if they're able to, you know, power on their laptop in the morning and they they end up you know, connect to the wifi and boom, they're able to do their job, that's the best possible outcome.
Now in the corporate environment, I think we do have the luxury that, you know, you can have a team of folks who, who do think about these things and you know, you, you have the luxury of being able to have, you know, tier one, tier two, and, and you know, like escalated support chains and all that. But I mean, for a small enterprise like, or like a startup, this is, this is a tough journey, right?
And, and I feel for some of the recent breaches that happened where it was because of, you know, challenging controls on, or minimal controls on laptops or, or whatever it may be for, for like critical SREs or whatever. I mean, first of all, great example of targeted attacks and all that to, to bear in mind.
But second of all, Unless you're willing to invest the effort and time, like are you going to be able to have that ideal posture? You know, in your 10 person startup, are you going to be able to you know, make sure all of your developer laptops are properly locked down or they're not using them to host, you know, some sort of shady service or something by night or whatever it is?
And, and I think that's, that's the, the tricky part, right? Like, I mean, the computers. Of today are so complex and that just the, the, the sheer amount of, of software that's running complexity in that software and, and how many, you know, individual services and components make up, you know, a fully functional whatever, macros or windows or whatever.
And, and I, you know, having that expertise in what each component does inside of even a corporation like Bloomberg is, is, is impractical. So I, I just don't even know how these days, how you reason around posture unless you have kind of deep pockets and resources to, to, to invest and, and you know, there's a lot of products out there that will probably help you find your way.
You know, like if you're managing a fleet of, of Mac laptops, Jamf is a fantastic tool that you could probably use as a, as an mdm. But you know, for your 10 person startup is. Is that gonna be something you're gonna want people focusing on when really you're trying to get the next sale or the next revenue, the next bit of revenue.
So there, there's an economic, you know, asymmetry here that, that I think we have to deal with. Mind you, you know, to kind of give a shout out to Apple I think they've done a lot to try to make their platform at least out of the box, as secure as possible. And it, it does interest, it entertains me sometimes cuz you'll, you'll see within the tech community, people.
Will, will be outraged. Like I bought my Apple device, I want to be able to run whatever software I want, but it won't let me do it out of the box. And, and all I can think is like, for you, that's a pain. But you know, if I can give my primary school teacher mother a Mac and know that it's gonna have by default a secure posture as long as system integrity protections enabled and all that, which by the way, how would she be able to know the difference between system integrity protection being enabled or not on a Mac?
I don't think anyone can solve that problem. You know, that's you know, I just, it, I, I think Apple's actually done a good job though of making sure, like out of the box they've, there's, there's a great deal of, of smart and reasonable trade-offs from a usability perspective for, for the security posture of the device.
Now that being said, obviously you get the, you know, user that samples sip or you get a, a user that installs a bunch of shady packages from God knows where. You know, you, you're still gonna have the, those same types of access problems or whatever it is. So it's, it's not panacea, but you know, I think part of it is like, as vendors like Apple push the limits as to what the default posture system is when you know, out of the box, I think that will benefit everyone.
But that said, it's a long journey and, you know, it's in the end, it's a device that runs arbitrary codes, so, How are you going to reason around what everyone's doing on it? So it's a, it, it's, I think this is like the, the challenging impasse we're at, right? It's like how do we ensure we can make these things accessible while also you know, ensuring people can get their job done.
And, and, you know, I, I can do that talking in the context of Bloomberg cuz we have a great team of folks working on this problem day in, day out and thinking about, you know, what are the next steps and what are the next innovations 2, 3, 5 years down the road that we have to think about. But not everyone has that luxury either.
Elliot: Yeah, I, I love that perspective. And I, I don't know if it's today or just recently, and this is kind of an ongoing thing, but obviously Apples have been duking it out with the Mio or UK market on, you know, how they lock down their iOS app store and now they're being pushed towards enabling side loading.
So obviously that impacts them from a revenue perspective, but, For everyone else and the user perspective, it greatly reduces the possibility. Malicious stuff comes in there cuz they actually review every single piece. So I completely agree with that. It'll be really interesting to see once those floodgate open, you know, are we gonna see issues like Android?
Or hopefully maybe they'll have some innovative approaches to keep at least a better secure ecosystem.
Phil Vachon: I, I yeah, it's gonna be very interesting to watch. I really hope there's an MDM control that allows us to disable side loading is all I can say.
Elliot: Absolutely. So yeah, I think we're kind of towards the end of our, our time, but you know, you have covered a huge use case for us, which has just been kind of that missing nugget that we've really been. You know, hoping to get insight from, from someone who has built that within an organization, but not just any organization, organization with you know, historic structure is clearly, you know, has just multiple facets of, you know, on-prem, off-prem, and everything in between.
So we really appreciate your perspective being able to come here and share some of that insight and hopefully our listeners will be able to garner some of that expertise and be able to apply it at their own organizations.
Phil Vachon: That's fantastic. It's been a lot of fun chatting with you, so thank you. And, and I'm always proud to share the work that we do because I think we do some pretty awesome research and, and honestly, some of the stuff we've built is really cool, so it's exciting to even talk about it.
Neal: Oh, thanks for putting up with my rabbit holes, Phil. Appreciate it.
Phil Vachon: No, it's okay. I can go down plenty of them myself, as you saw. So.
Adopting Zero Trust with Bloomberg: Implemented