S1 E3 US Digital War Biometric Watchlist Iraq Afgh

US Biometric Watchlist Digital War Iraq Afgh

[00:00:00] Thank you for joining me today. I'm Bill Alderson with Disaster Stream Podcast. We're coming to you from beautiful Austin, Texas. Right behind me is Lady Bird Lake and downtown Austin. And if you want to have a good time, come to Austin, the live music capital of the world. I've got a great responder story for you today.

I got a call. I got a call because the United States military had a problem with one of their [00:00:30] digital war fighting systems. It was an intelligence system that used biometrics and it was key to maintaining and having a watchlist of the worst insurgents in the wartime effort.

First contacted by the G-Men actually, Army G2 at the Pentagon. Flew back there, got briefed on what was going on, and then, talked to various folks, JOINT, CHIEFS of staff. Then we went down to [00:01:00] US CENTCOM, where we got outfitted with the rest of the team to take us over into Iraq.

They said, would you mind if you deployed with the troops? And I said, it'd be my honor. And the rest of our team, they were excited about it and we all jumped at the opportunity to fly to Iraq and help solve this incredible digital war problem. I'll talk to you a little bit more about it, but [00:01:30] before I do, I want to make sure that you all remember that it's definitely more fun to be ready.

And that is the message that I am talking about today. We were ready to go to Iraq to help with the problem, and we want you to be ready when you get that call or your company calls upon you in time of need. Here we go.

I call it disaster stream. It's on our website is [00:02:00] disaster.stream. That's it. Just disaster.stream. Put that in your browser and you'll go to our website where you'll find all of our podcasts and other information. This is season one, episode three, we're brand new. This is something new that I've been developing for the last six months or so.

Stories about where responders have to respond to a particular disaster. Mainly these disasters are of a [00:02:30] nature that affects data because that's where I have spent my entire career nearly 40 years now, looking at packets on the wire, diagnosing critical problems leading teams to solve some sort of a problem or help recover from some sort of a disastrous event.

We've got the US military in Iraq and Afghanistan, and the digital war that they were fighting there, which a lot of people [00:03:00] probably don't really know, but we have been fighting predominantly a digital war. How do we do that? We have insurgents and it's very difficult to identify your insurgents or to identify the average person in a community versus that of an insurgent.

What the US military did was first, the Marines took this system and they went to Fallujah, which was the hometown [00:03:30] of Saddam Hussein. They went to that town, they made everyone leave, and then they burmed the whole city so no one could go in or out. And they had three points of ingress egress.

And when somebody went in, they had to give their fingerprints and an IRIS scan and other identifying information, and they built a small dossier on everyone in that particular area. Later on, you're going to see why that was important. But when they got to [00:04:00] 1.2 million enrollments into the system, it stopped working.

And that's why I got the call to go in and figure out why this incredible biometric intelligence system stopped working mid war. We went in and started taking some action along with USCENTCOM.

Part of our mission is to tell your story. I talk about my stories as an [00:04:30] example so that you can see how your story might fit into our formula. What we do is we try to tell a story from the responder's viewpoint and what happened, and then we pick out the lessons learned, the things that we did really well, and the things that maybe we didn't do so well.

Those are also lessons learned. We can become more ready to respond to a disaster professionally. If you have a story, your team has a story or [00:05:00] someone you know has a story, let them know to come and watch. Disaster recovery responder stories here, and we will show them and talk to them about being part of our broadcast.

On every broadcast. I like to talk about other people that I know and work with that have spoken at our conferences or I've worked with in the past. I have two people today, Charlene Deaver-Vasquez of FISMACS. [00:05:30] She's going to talk about something really cool and you're going to want to understand this because it's a, takes a very short period of time to understand something that might really provide some very positive understanding of what's going on in the mathematical models for forecasting Cyber attacks.

What I do is I just queue up a promo of her session.

Her session is about 30 minutes, and she goes through and teaches on mathematical models [00:06:00] for forecasting Cyber attacks. I give you the link in the show notes to about a 30 minute piece that she did for us at our recent Austin Cyber show.

Another treat that we have for you today is Jon DiMaggio. Jon DiMaggio has written this great book, the Art of Cyber Warfare. Jon has looked at nation state Cyber threats, and he has some cogent information for you.

He'll [00:06:30] introduce himself. And then I will give you the link so that you can listen to him for about 30 minutes, talk about insights of his experience doing nation state analysis. And then, of course, his book is a great resource to add to your library.

Out of my lessons learned and over my career, I have had to respond to a lot of data crisis, a lot of disasters and respond . These [00:07:00] are a list of some of my problems that I have gone in and disasters that I've had to help recover from. But I want to just point out how relevant is this? It what lesson learned from the past might have prevented Facebook being down on October 4th, 2021, just about a year. Where they lost full connectivity for their entire organization for over what, four to six hours. It was a disaster. They [00:07:30] lost about 5% of their market value, which was about 25 to $50 billion. Now, it doesn't take long before not having that occurrence and saving 25 to $50 billion in market value, somebody lost that market value on that day.

It popped down, and then of course, it may have gone back up later, but somebody lost 25 to $50 billion. [00:08:00] Why? Because there was a lesson learned, a best practice that Facebook didn't implement, and it cost them mightily. I will talk about this in a future session. I always I'm basically wanting you to understand that even though some of these things, Pentagon was 20 years ago, some of these things were in the past, but the things that you can learn from the past can save you millions that are possibly billions [00:08:30] of dollars in the future if you are prepared.

And that's my message. I usually go over some little tidbit of a disaster recovery timeline. I'm not going to do that today, but as I introduce myself and what we're doing, I talk about the timelines. I talk about tiger team formations, how to document and peel the onion and do analysis.

So I, I'll always bring to you a little bit on that. This is just boilerplate that I go [00:09:00] over and I'm just introducing the topics and the ideas so that you get an understanding of what we're talking about on the show. . Now, the main purpose, once I find and identify lessons learned, then we want to move that into a best practice.

And those best practices often are not easily implemented. You may need some help. Maybe you're a large Fortune 500, fortune 100, or a government institution or a nation, while you [00:09:30] still want those best practices, but it's hard to get those things imputed into your organization. If you take a look over on the left, I've got the high fidelity low noise input that you want to amplify.

That's why the center of this shows a little transistor amplifier. And your organization is going to amplify the results of those best practices. And that's what we're talking about is imputing and applying those best practices. Sometimes [00:10:00] you need some help if you're a defense contractor. You might want to go out and look at Cap Gemini general Dynamics it, other consulting organizations like McKinsey, et cetera.

 Here's today's story. The US Iraq, Afghanistan Digital War, using biometric watchlists failed at a very critical time, now authentication and biometric recognition [00:10:30] identification. It's very difficult in a wartime situation to take the population and identify people. They don't always have driver's licenses with fingerprints or any other sort of definitive, and the thing that is definitive about identification is your fingerprints, your IRIS scan, facial recognition, and other such things.

So the military used these in building this particular capability out, and it's very [00:11:00] powerful. We're talking about taking Fallujah, for instance, and the Marines took the city of Fallujah, which was Saddam Hussein's hometown. They made everyone leave and then they burmed the city so that only three points of ingress egress were allowed.

And marines were stationed at those ingress egress, and they had to submit to getting a biometric identification dossier built on every person going in and out [00:11:30] so that you had fingerprints. And the whole purpose of this was so that we could identify the insurgents by biometric information.

And I'm going to show you in subsequent slides how you can actually find an IED bomb maker by finding their fingerprint and searching through those fingerprints and finding that needle in the haystack. And that is how the Iraq war surge was so successful is because they had this tool [00:12:00] to identify out of millions of people who were the bad guys who were on the watch list.

And I'm going to go through and show you how some of that happened. Now, fingerprints, IRIS scan, they're coming up with palm prints, they're coming up with various facial recognition. There's a lot of various biometrics in this system allow you to put in almost any type of biometric, and they were adding to it on a regular basis and most likely still are.

This is the [00:12:30] biometric ID path to identify and find and catch a bomb maker. Up here on the top left, you have an IED that exploded and killed some people. While some of those fragments of that IED still have fingerprints or other identifying information, maybe a hair sample or something of that nature, Or they found a place where they were making those IEDs and they scrambled away and left, but they left some [00:13:00] incriminating evidence about the fingerprints of the people who were there making those IEDs.

By taking the general population and taking their fingerprints and IRIS scans and then scanning back, what happens is after they find those IED fingerprints, they do CSI work. You watch television, you can see the CSI work. And so they go in and they recover hair, DNA fingerprints off of [00:13:30] maybe some tape that was used to tape up the electrical components of a, of an IED. And they take those biometric components and they bring them centrally and they communicate them across large networks. And this is what was happening. I'll explain, a little bit more, but basically by identifying all of these people we can bring together and come in and we can win the digital war.

And it's an enablement to be [00:14:00] able to find bad guys from their fingerprints, IRIS, scanner, other identifying information that's definitive. Intelligence systems go out. Identify that the, there was a cell of people making bombs in a certain area, and then they go in, take fingerprints, send it to the CSI lab, they're right in country.

And then they bring all that data back to intel analysts and they put everything together. [00:14:30] And pretty soon were able to match those fingerprints that were on that IED to somebody that we did an enrollment. And then when they come through a security checkpoint, they put their finger on the platen boom.

We know that this person was a maker of an IED and that's how we prosecuted the digital War in Iraq. Pretty cool. This is the system. You take a variety of inputs, fingerprints, IRIS, scan, [00:15:00] pictures, other such things, and you build it into a system. Now this is a diagram that I made so that I could understand it a little bit better and then once we had an enrollment with a dossier that would go in and hit a SQL database in various files, those files were then, all around so that people could share those particular enrollments.

They also went back to theUSto the biometric fusion center and [00:15:30] even back to the F B I, so that people could see what was going on with this. And then, but this system got overloaded and it would replicate between all of the various servers all over the world that was involved with the war effort across Iraq and Afghanistan.

And it stopped being able to replicate. That's when I got that call and said, Bill, would you go over with the troops and go to Iraq and help us figure out what is wrong with our [00:16:00] system? Why is it not working? Why did it stop working mid war? I'll show you a few things. As you can imagine, things were, these are enrollments in Iraq, in Afghanistan.

There was a lot more enrollments in I Iraq than there were in Afghanistan at that particular time. And then there were other parts of the worlds that we were taking these enrollments as well. And I took these statistics and I put them into, A [00:16:30] chart so that I could see what was going on as I worked on this particular system.

And here's yet another one a similar kind of chart, but it showed you the enrollment by source. How many did SOCOM do? How many did BAT do in various biometric enrollments across the entire war area of the US military? And then they would also keep track of how many latent matches we [00:17:00] were getting and how productive the system was at identifying and putting people on the watch list so that when they would come through a checkpoint boom, or we had a suspect, we would know that they were in this particular area.

They also used this system for things like vetting who could go to work in, on a military base because they used Various TCN, Third Country Nationals, and they used national Iraqis and [00:17:30] Afghanis to help run the military bases. They did, labor, they got paid, they work, like a regular job.

But every day they would let them come onto the base and then they would leave the base and they would use these biometric systems to allow them on and off the base. Also, the military used these for base access so that they knew that it was definitively a military member or a member of the institution being [00:18:00] allowed in and out.

So we analyzed all the biometric systems. Now I want to talk to you a little bit about the workflow. Whenever you're trying to solve a problem, you have to have some telemetry data. you see at the top in the blue, you have strategic instrumentation and various metrics and systems that you have to prepare for in advance.

You have to instrument your environment. And that's what we spent a lot of time doing is [00:18:30] instrumenting the environment. We instrumented the entire war network in multiple locations around the world. And then we would take the findings from that and we would break it down and we would figure out problem by problem, issue by issue.

Illogical findings. And then we would discuss, troubleshoot, test, make recommendations. And it was a continual improvement program to solve the problems. And then we would [00:19:00] record all of our findings over time so that we could make certain that we. Use these new lessons learned continuously. that's how we did the work over there.

I developed this methodology over 40 years of doing critical problem resolution. And this is how I work. I peel the onion back, I do analysis. And the problems are many. There's not usually one problem. There's usually a myriad of problems. that's [00:19:30] why I call it peeling the onion. You peel the onion back on the first problem and then you incrementally break it down until you get down to the heart of the problem.

That's the most material problem because you can always find problems. The thing is that you want to find a problem that is causing the most pain. That's what this is designed to do. Now here's some pictures of me and some of the team members. When we were going over or coming back I made six trips to [00:20:00] Afghanistan, Iraq, Qatar Kuwait Bahrain, Djibouti, you name it. I was in all of those various countries. They had various communication systems going in and out. And then I also went to all the locations around the US where the data would flow into. To have that intelligence analyzed .

And we analyzed every step from the war fighter putting in the initial fingerprint and doing the initial enrollment. [00:20:30] And as that enrollment would move through the system and then replicate around the entire world. I was analyzing all of those different things. And this is just a couple of pictures.

This is the, a Al Faw. This is where Saddam Hussein was often located and his sons Qusay Uday, his, their houses were nearby. They had a lot of these palaces, I think over a dozen of them, and they were quite palatial. Here's SU Saddam Hussein's big chair that was out in the [00:21:00] middle of the area.

And we would all take turns to sit down and have a picture there.

And here you see I was in a C 17 and we had a huge this actually is a picture of me inside of C 17 and I was joking around because it was a big power generator. And I was saying, Hey, here's how I can hook up my my razor to this system. Anyway, little joke. And we had marines, we had Army, we had Navy Air [00:21:30] Force people on the team.

We also had civilians. We had quite a large civilian population who came with us. We had, each time a colonel would lead our 10 to 20 people who would go into country to do a lot of these various analysis things. And that's and help us accomplish the goals. The colonels would then brief the generals as we were coming through and what work we we are going to do on which trips.

And it was very well coordinated by USCENTCOM who led [00:22:00] the entire operation. We had to instrument the entire Afghan, Afghanistan, Iraq, Qatar, Kuwait, Djibouti, CENTCOM, all over the world. We had to put systems in that would monitor the network and that would watch these transactions as they went from the war fighter through the intelligence systems replicated to other servers inside the AOR, the area of responsibility, [00:22:30] that's a name that they, the US CENTCOM was responsible for both Iraq and Afghanistan.

And there was a lot of instrumentation we had to set up in advance. This is what that looked like. Okay. Now like I said, as I go through, I like to introduce you to some of the folks who I work with or that I know and have spoken at our various conferences. And here we are going to talk to, and listen to Charlene talk about mathematical [00:23:00] models for forecasting Cyber events.

 Hi, and welcome to this session, mathematical Models for Forecasting Cyber Attacks. My name is Charlene Deaver- Vasquez. And there's actually quite a lot that I want to share with you, I kind of want to dive right in and just get started. Some of the things I want to cover in today's session is an overview first of the mathematical methods that we use for [00:23:30] estimating probability of Cyber attacks and that we use in some of the models. And also I want to cover some of the mathematical model use cases.

And then I want to talk a little bit about what the analytical process itself is. And then at the very end I want to be able to share with you a brand new model that will give you a glance of what the future of these kinds of models might look like. It's based on some math that was actually only theorized a year ago.

So excited [00:24:00] about that. Let's get started.

Okay, here we are. We're back. Thanks for listening to Charlene for a minute. I hope you'll go back and check out her longer video and learn a little bit more about what she's talking about now. Remember how I told you that all of these servers and systems would replicate around the world? There was a problem with it, right?

We'd had to replicate that watch list and that system around the world, and this was a picture of some of the servers [00:24:30] and how the replication would work and how many hours it would take. And we had to calculate all that out. How much data was moving back and forth to figure out. For the Intel people from the time a new enrollment came in, how long would it take it to get to the us?

How long would it take it to get to other areas that we would know if it was 10 hours, 20 hours, 72 hours from the time an enrollment of a bad guy until that [00:25:00] watch list was replicated around the world that if they, we encountered them. Of course, now just thinking about this would be a very good thing to have at our southern border right now, as we have thousands of people coming through a day from 150 different countries, we ought to have them put their finger on the platen or the IRIS scan that we know if they are on the terrorist watch list or not.

I think they do that, and I wouldn't be surprised [00:25:30] that was how they were catching a lot of these folks. now, when you replicate from server to server, it says, first of all, what do I need to send to the other side and how long does it take to get there across the network? we're talking about seconds here of going back and forth.

So it would say I need to have all the new updates. And that would take a certain amount of time, and they were a certain amount of Bytes and that's what we were doing. We were analyzing how all of this [00:26:00] stuff worked and how long each one of these processes would take. I broke it down.

To, each individual software function, fetching the keys, send the keys. Now the keys are the differences in the entries in the database. And then it would say, I need this many rows of data. And then it would send that data repeat and it would continue on until the whole database was replicated to another [00:26:30] server in another area around the world.

And then that w that server would be updated and then that server would update another server. Very complex system. And we had to time each one of these things to determine why would not continue to replicate once it got to a certain size. we had to figure all of these things out that we could then go back.

Ultimately to Fort Huachuca , the Army's Communications Center [00:27:00] in Arizona and helped them rewrite the code. And that's what I did. I went to Fort Huachuca . I took all of this information and I helped them rewrite the code and then we tested it. We had a huge lab there where we sent example traffic simulation, traffic, simulating the satellite links and all the various links and simulating the replication across the AOR only we did it in a lab [00:27:30] that we could see if we were improving things, where we were going.

It was a very interesting, very capable, had some incredible programmers who would improve each one of these. So here's just some views of time. this is time. Each one of these are packets. The packets would come through, it would process the data, it would send the packets to the next server, but there were some problems that would happen.

There's what's called packet loss [00:28:00] and packet duplication. In other words, they were sending the same packet multiple times. It was not productive. And you can see that it would basically stop right here and it would slow down the the ultimate completion of these replications, which would take many times, well over 24 hours to re.

So that's what we were doing there and that's what we were analyzing. And I have some more details here I'll show you. I know that you're not a necessarily a [00:28:30] technologist, but these pictures help you understand what your technologist should be looking at. we missed two packets here, dozens of retransmissions, you can see the same packets sent twice.

I call that data duplication. In other words, you're using the bandwidth twice or three times or even more. I call that data duplication, where it's not just retransmitting something that was lost, it's actually sending the same data multiple times. And of course, that can [00:29:00] contribute overall to congesting all the links.

And then also in increasing the time to finish the synchronization. . Now, as with any problem, you first quantify what's my problem? What do I have? And I use the difference between a square and a cube, and I'm going to define this here for you, A disastrous problem, something that I have diagnosed my [00:29:30] entire 40 year career.

I come in and I take a look at. What the problem is, what the environment is, what the team is saying, what the symptoms are, what the reported symptoms are, and they're never always the same. There's a lot of different conjecture, and you cannot solve today's problem with today's information, otherwise you would solve it.

Right? There's something that is missing, and many times it's a different analysis. It's a different [00:30:00] viewpoint. It's a different perspective. that's why I use the difference between a square, which just gives you one, two dimensional piece of information. And then I compare that to this, which is a cube, which gives you three dimensional information.

And that the, that third dimension is a new input, something new. There was something missing in the disastrous problem [00:30:30] discussion or the findings. There was something that was not understood about it, and in order to solve the problem, we had to have some new information. Once we are equipped with some new information, yesterday's problem can be solved with new information, and that's what we call a paradigm shift.

A paradigm shift is yesterday. I couldn't solve it. I knew what the problem was, but I didn't have the [00:31:00] key understanding of what was going on. And then today I have some new findings, new visibility, some new knowledge. I have a new expert. I have a new piece of forensics. I have new diagrams. Illustrator, help me understand.

I have a new metric or some root cause analysis that helps me understand and get a payoff. In other words, the new data allows me to solve the problem. And it's actually typically [00:31:30] quite simple. It's a little disappointing because when I go in to solve a problem, it's a square, nobody knows the answer. And then I come in and I solve it, and I diagnose it with new information or analysis, and I figure it out.

And we have a paradigm shift. And sometimes the paradigm shift is rather simple. It's some new piece of information that told us what the problem was. And then we took hypothesis, tested it, the new improvement, tested it, and [00:32:00] validated that it actually was in the the solution. that's what I've been doing all my life.

So that's why I have these kind of views. Now, the response time from the servers, remember how I showed you the diagram of where we put instrumentation all around? We put that instrumentation in order to do server response time. when a packet comes to a server and then there's a response from the server application, we would classify and figure [00:32:30] out.

How much of the time it was just sitting there listening and calculating what the answer was going to be, how long it took for retransmissions, how long it took for the data to traverse across the speed of light and back and forth with the various protocols and systems. And we would calculate this out so that we could visualize and see when the response time went high, maybe it's due to congestion.

[00:33:00] There's a lot of requests coming in, and they're queued and there is a queue depth and it, and there's a response time delay. you basically come up with these response times what I call a rule of thumb or a general rule of, hey, 449 milliseconds. Now when packets are going across a network, it's about one millisecond per hundred miles of distance latency.

That's. That's not just a good idea. It's the [00:33:30] law. It's a speed of light. when you calculate all these things out, if you're going to send a packet, for instance, to London from the US it's probably, 5,000 miles. you go 5,000 miles over, that's 50 milliseconds and 5,000 miles back, and that's another 50 milliseconds.

you are going to have a hundred milliseconds roundtrip delay. it depends upon how far you're going because of distance latency or [00:34:00] the and then there's satellite delay, which I'm going to talk about because satellite delay can be incredibly significant. And now we have these, new Starlink systems that Elon Musk has launched.

He's got thousands of low earth orbit satellites. Most satellite communications have to go down to a satellite that's on the equator that's going around the equator. And of course the equator is the longest distance around the us. And[00:34:30] these satellites are at the equator, and they're 22,500 miles to the equator and 22 500 miles back down.

So it would take, 125 milliseconds each way. 250 milliseconds just to traverse a satellite link. And that's a lot of latency and that's why you can't really use a telephone across a satellite link very effectively because there's a lot of latency. [00:35:00] It's kind of like you say something and then you have to say, over, and then you have to wait for the response.

And it takes a lot of time because of the latency. When you have low Earth or orbit satellites, those satellites are about a hundred miles straight up above you, and they're very fast, right? Because you just, you don't have to go 22,500 miles down to the equator. And that's why Elon Musk Starlink is brilliant.

It's just, it's they're very simple [00:35:30] physics. And here with the Ukraine War, we deployed those and very powerfully we're able to continue to get Internet service across the Ukraine, even though the Russians destroyed all the infrastructure. Of the terrestrial lengths on the ground.

Okay, that's a little bit about that. Now, when you're analyzing these packets I developed some of these charts and I [00:36:00] basically just helped people who weren't technical like me understand what was happening and the delays and various things, and what happens every time there's a packet loss.

These things would happen in the recovery of the TCP/IP protocol oftentimes was not very efficient and it would actually cause a lot of time to recover, or it would retransmit the same data multiple times, [00:36:30] wasting bandwidth. we ended up having to visualize this for generals and other folks to understand, and then we had variations of variables that we could change on the servers that would affect and improve that for certain things.

And we had to visualize this. I took this and built these charts and graphs . Other people who were not technical could understand what I was talking about, because I don't need those [00:37:00] charts. Now I like them because I can very rapidly see and explain, but I don't need those. When I look at a problem, I know what the problem is and I can say, Hey, you need to do this.

But other people, they have to spend money they have to change product, they have to change protocol stacks, they have to change applications. you have to help them understand. And the best way I know to do that is to help people visualize a problem. that is a visualization of data duplication across a [00:37:30] TCP/IP Internet type circuit.

The other problem that we have is you have a lot of interfaces. You have a router and then you have a satellite, and then you have a satellite, and then you have another router, and then you have another satellite, then you have another router, then you have a terrestrial link. And many of these circuits.

because it's a wartime network, would have problems and they would have errors. we would count the errors. We would quantify the errors, because here we [00:38:00] are in a noisy environment, bad cables or cables that were longer than they were supposed to be. By necessity. They had to build cables that were a little bit longer and we ended up having interfaces, having errors and drops .

And that's why we had to show facts. These are scientific facts about why it was slow. So that if you want to mitigate that, you have to [00:38:30] fix that by spending some money or changing some parameters or doing some study. . Here's another example of the very same thing. How often it occurs, where they're occurring.

And then that way you can go to those devices, you can go to those systems and fix them more efficiently if you know where problems are occurring. And this was packet loss and poor recovery. TCP is supposed to have the Internet. Every time you use your browser, you click on something.

Sometimes you keep [00:39:00] clicking because it's slow. Sometimes that's because there's some error on the Internet that's causing your packets to be discarded because. There's either congestion, too much traffic offered to the same router, or there's some physical error somewhere, or some link in this huge worldwide network where it's broken.

So I take these packets, these are packets, one by one, and I show how come they're delayed? What's happening? What is the protocol doing? [00:39:30] Again, to visualize for people who are not technical for the chief network engineer or the chief financial officer, why do we have to buy these new circuits?

This type of information helps people understand why they have to do this. You don't, as a technologist, just say, trust me, I'm brilliant. No, it doesn't work that way. . You have to help people understand and begin to trust you, and the way that you do [00:40:00] that is by showing them. Scientific facts and showing them the delay, what's happening, the knowledge of the theory, the knowledge of the operating, the test equipment and systems, and then describing what's going on so that people who are making decisions about where to spend the money, where to spend the time, where to spend the energy, have the confidence in your analysis to be able to say yes and verily, this is what [00:40:30] we need to accomplish.

This is very true in the security world. You can't just guess at these things. You have to show people facts, figures, scientific information so that they can trust what it is that you're doing. Now, over there, in these environments, we had a lot of low speed lines, and those low speed lines slowed things down.

You can't put 10 pounds in a five pound bag. the military bought these Riverbed WAN [00:41:00] optimization, gizwatchies, and they basically put one-on-one into the circuit in Afghanistan and the other, on the other end of the circuit in the US and then they compress. They do a variety of latency and throughput optimizations that you hopefully get, you can put, more, you know how like on your disc you can encrypt or you can compress your [00:41:30] disc it uses bit compression.

That's the sort of thing that these devices would do only on a link so that you could get twice as much, three times as much. Bandwidth out of the same particular link. The problem was that several things happened. The packets would go from the war area to the US across one path, and they would come back across a different path and they would not hit the same link.

So in order to [00:42:00] use these WAN optimization methods, you have to send your packets across to this other one. They do all this magic compression, et cetera, and then they decompress it on the other side. Same thing when they're going back the opposite way. The trouble is that if you send your packets over here and they go.

and they don't hit the other device. They can't decrypt, they can't decompress, they can't do the opposite job of the optimization and then send the packets [00:42:30] out. they were sending a lot of packets across the network because of asymmetrical. In other words, it used a different path in one direction than it did in the other direction.

They were not symmetrical. consequently, there were a lot of problems because of this, and I was showing them here how I can prove that in the analysis here. Again, same sort of stuff, visualizing the TCP protocol visualizing this, [00:43:00] the selective act process, visualizing the window size and that sort of thing on a TCP circuit.

here are some more enrollments. This was how many enrollments they had in a day, right? Enrollment volume, how many Bytes by country, Bytes by protocol. How did all this? I took all of these statistics from the instrumentation that we put in so that we could understand what was happening out there [00:43:30] on the.

So that was the enrollment volume, a thousand people a day or 4,000 Bytes, and how large was an enrollment . Okay. Now this is basically a very simple thing. here you have your war fighters and Falluja, Mosul, Kabul, et cetera. And they were using various types of links, CENTCOM links, dis links, email, various types of communications [00:44:00] methodologies to get these over to the biometric fusion center, the FBI, where they did additional analysis .

And then they would come back and forth using various communications methodologies, and they used a variety of sec of security networks . you have the red and the black and th this is very complex, but basically, you're trying to get your enrollment from war fighters in the [00:44:30] field back to the FBI and biometric fusion center and other intelligence resources.

And this was over inside the war area. And this was in the continental United States. that's basically showing how communications, and then I'm going to break this down and see that there was severe packet loss in this part of the network, which is from the war network over to the US. Now, inside the US we didn't have that many problems [00:45:00] because we have good networks.

It's not a wartime network. It's not, sand in all the servers. There's not all sorts of problems. And then this is end-to-end communications and what kind of problems we had with satellites in, in, in Bagram and Baghdad and different locations around the world. And then this basically helped them understand where their problems were most material and helped them make better decisions.

All right, [00:45:30] now where we've come to the part where we are going to listen to Jon DiMaggio for about a minute, he is going to introduce himself, tell you a little bit about himself, and then you are going to, down in the show notes, you'll have links to Jon's 30 minute session where he is going to talk about the art of Cyber warfare and nation state financial attacks and other types of.

Cyber problems that Jon has analyzed. he is going to help you understand a lot of that firsthand. let's listen to [00:46:00] Jon.

 I spent about the first 14 years of my career working for one of the government intelligence agencies, and I was really fortunate.

I came in at a time where nation states were really starting to put together the programs that eventually began to attack the United States using Cyber espionage campaigns. So I got to spend the first, better half of my career really digging into nation state espionage actors, and learning a lot along the way.

Since then I've transitioned into the private sector. I've done a number of [00:46:30] investigations many of them. Things that have been, on newspapers, headlines things that you'd be familiar with since then you can see on the right-hand side, this is just the past year, some of the ransomware research that I've done.

And I've also recently authored a book called The Art of Cyber Warfare, which we are going to talk about some of the content today as well as some of the research publications that went into the. So everything today, don't worry, it's not a sales pitch for my book, but the book revolves around threat intelligence.

So that's really what I wanted to talk to you about today. And with [00:47:00] that specifically some of the objectives that I wanted to convey are why you need to treat advanced threats differently than you do traditional day-to-day Cyber threats.

Okay. Thanks for listening to Jon for a minute . Now I'm going to talk to you about some other problems routed. Networks like the TCP/IP network, like your Internet. Sometimes things are changing out there in the Internet proper. We typically, in the US have very [00:47:30] good networks, very low errors. Very low packet rate packet problems packet errors, packet loss, .

But at certain times there's more or less one of the attributes is in order to update how big the Internet is or how small it is, we add networks in and they're constantly having ads deletes and changes, and that's occurring. And we have millions of networks that are all connected. [00:48:00] Routers are updating where, what network IP network numbers are going where?

I put a system on here called the BGP activity summary, showing the number of changes. So you can see here that we would have what I call churn. You'd have sometimes 10,000 changes in a very small period. Now inside the. These are usually very small down here. But once in [00:48:30] a while we'd hit a window where there was an incredible amount of churn.

And here you can see when networks were coming on, new networks were coming on, new networks were leaving. So you have withdrawals and you have updates, and that all contributes toward the changes in all the routers. And that's why packets would go over in one direction and come back in another direction defeating the ability of these wan optimizing [00:49:00] capabilities because the packets wouldn't come back the same path.

And that was a big problem and they had a lot of those things because they were trying to use lower speed satellite links. Now when they had true. Drone operations or weapon systems, that was a different network. This is mainly communications for the war fighters, thousands of war fighters. it was not as real time intensive [00:49:30] as flying a drone or something of that nature.

And they'd use a different kind of network for that. But this is just showing that you those changes, and this just shows you that there's a lot of routing metrics changing, and I have some examples of this. a packet comes into a router, pops through, goes to another router, and then it pops up 22,500 miles to a satellite that's on the equator, and then it comes back down over here.

And that's going to take a lot of time. [00:50:00] Now, if that were. A 500 mi from camp A to Camp B, it's only 500 miles. It should be five milliseconds, and yet it's 250. What was the problem here? In Iraq, Afghanistan, and other such places, the war destroyed all the fiber optics and a lot of the terrestrial systems so that you had to depend upon satellite communications in order to communicate even, [00:50:30] shorter distances.

And that increased the latency in a lot of our applications, no matter what it is, waiting a half a second for communications and you add that up if you have to get something 10 times. And. A half a second every time. That's five seconds. it ends up being a big problem. this just highlights that.

That's just one satellite link. what I want to show you is that because of some routing inefficiencies, [00:51:00] instead of going just between camp A and camp B, it would go from camp A to camp B, to camp C to Camp B, and it would cascade those problems and it would double the latency or triple the latency and go 90,000 miles with a packet instead of the 500 miles that you would normally experience if you had a terrestrial network under normal operating conditions.

Okay, now remember, [00:51:30] The if during the Iraq war and I and and the Iraq Afghan Wars, we would've had the satellites, a low Earth orbit satellites that that Elon Musk launched with they, they don't have to go up to 22,500 miles to the equator. They go straight up, hit a satellite, and then there's a, an array instead of one satellite that's sitting there on the equator.

And you go up and back and up and back [00:52:00] to the same satellite. These are an array of satellites around, they call them low earth orbit satellites, and you go up and it's like maybe one or two milliseconds, and then they go boom, around the world, and then they come down. you have satellite to satellite communications, and it's almost as if.

You had this type of terrestrial links and you don't have to go under the ocean, you can just use the air up inside [00:52:30] the atmosphere or up inside the just beyond the ionosphere out there. We have better technology now, but these were the existing technologies. So we used all of. We deployed all of those satellite links, but it took a lot of time and we ended up with routing errors.

We would compound our problem. And so now you can understand why we had some of those particular problems. as we went through, we would find these problems and we [00:53:00] would improve and it lower the latency, lower the number of sessions, we would improve it. And we would basically look at quantifying the improvement.

So we had a big improvement, then another improvement, and that's what we're basically looking at is incrementally improving things as you have a problem, you have a diagnosis, you have. A fix. And then you start putting those [00:53:30] fixes in and sooner or later you get down to where you've got a very low, see that green?

That's the best response time you could get are the best number of sessions. And here you've got the best situation that we possibly could have, and it's improved by orders of magnitude. Okay, what lessons learned did we have? We ha we had a quite a few lessons learned that if we went back and fought another war, Or our military wanted to upgrade some [00:54:00] systems.

We already know what we could do to improve operations pretty significantly. And it's only been actually, I think I was over there in, we didn't end the war until last year. And consequently, they were still using some of these systems. They had. Fully replaced. But if we we're going to fight a new war, we would probably do it and architect it a lot differently.

And you would take the lessons learned from this war and you would apply them to military [00:54:30] communications in the future to improve all of these things. So we wouldn't have asymmetrical network paths. We would have the same path coming and going so that we could use WAN optimization equipment. We would watch for the continual churn, and we would not have flapping routes and flapping packets and packets would go over and then stop because they had nowhere else to go.

And they would spin [00:55:00] inside the until their Hop count expired. And there was no convergence. There was. There was no just quiescent happiness. Things were always changing dramatically. And then of course, satellite delays. We would've probably designed our satellites to be more like low earth orbit satellites that Elon Musk is using with Starlink, instead of these satellites that have to go 22,500 [00:55:30] miles in each direction we would have better battlefield communications and we would have better instrumentation because we learned a lot by instrumenting these things and so that we could find the heirs.

Now we know how we need to instrument so that we can find the errors and then solve them rather quickly. Rather than just suffer and suffer for months and months or years with the same errors that were occurring. We can find the errors, solve them, mitigate them, [00:56:00] and move on and keep. An error free environment.

Okay. And one of the problems was that they threw all this stuff up and it was thrown up by different people and different organizations, the Army, air Force, Navy, Marine, different battalions, different things. And that it was very difficult to have the network architecture documentation. And that's one of the things that we worked hard to try and help USCENTCOM take care of was the [00:56:30] network architecture documentation and automating some parts of that so that it wasn't rare and always different.

It became more symmetrical, more reasonable. And then we made sure we had diagnostic tools and we would ensure that we had good technical training for war fighters. There's a lot of lessons learned that can save. Billions and billions of dollars in the next war, or the next war effort or the next incursions around the world.

Pretty [00:57:00] cool stuff. All right, this is a picture of our fearless leader. There is on the second from the from the right. That's Colonel James Kirby. He took us out there and he was the one who brought us through the first time. And Ben Kohler there on the right. Steve Mercklein in the center myself.

And then we had major, I can't even pronounce her name, but she was a Marine and she was pretty tough. Okay, [00:57:30] that's a little bit about what happened now, the next. Time that I went a total of six times, went one time with Colonel Kirby, and then came back and got paired up with Colonel David Wills, who I went on the next five trips across the next several years over to Iraq and Afghanistan and other points around the world.

We talked about him in, in one of the last sessions that I did, and I'll include the link so you [00:58:00] can hear his keynote address. Hope is not a plan and I'm not going to put his His little intro here, but because I've already done that in a previous broadcast, but you can find his link to his talk in this particular thing.

So now I've gone through the stock market denial of service. I've gone through the Pentagon 9/11. This is now the US Military Biometric Intelligence Application in Iraq, [00:58:30] Afghanistan, and around the world. I have a whole bunch of others with the federal agencies, everybody from the I r s to the Department of Justice, to the department of State energy companies, financial companies, telecommunications carriers, other fortune 500 data disasters, problems that Cisco had with their products or other companies had with their products that I diagnosed.

Anyway, I'll be going through those particular things as we [00:59:00] go. Remember, if you're one of the responders or some of the responders or your team responded and you did some really cool stuff I want to hear about, I want to interview you on the show and then pull out the lessons learned so that we can all learn.

we are going to hear from a 9/11 New York responder somebody who was working with all the trading floors or the trading companies in a little bit in the future, and I'll so be on a lookout for that. [00:59:30] All right. thank you for joining us today. I really appreciate it. You can go to Disaster Stream and look at a little bit more, watch other episodes.

Thank you so much for being with us today. We really appreciate you hanging out with us.