Episode 84

full
Published on:

14th Nov 2024

Datacentres

In this week's talk, Amit and Rinat dive deep into the world of datacentres, a location where all the data we create or consume gets stored. We break down the different types—from enterprise and colocation to managed service and cloud data centers. We also compare the nuances between on-premises setups, managed services, and cloud solutions, helping you understand how businesses choose the right infrastructure to power their operations in a fast-evolving landscape. Join us in this exciting conversation.

Transcript
Speaker:

Outro Music

Rinat:

Hi everyone.

Rinat:

Welcome to tech talk, a podcast where Amit and I talk about all things tech.

Rinat:

We don't just talk about tech.

Rinat:

We also talk about how the tech impacts our lives and the society altogether.

Rinat:

This time we're going to talk about data centers.

Rinat:

Data centers is one of those topics that , all of us have heard off.

Rinat:

And even if you didn't, you could probably guess what data

Rinat:

centers are anyway from its name.

Rinat:

There's so many things I know about data centers.

Rinat:

but as soon as I think about talking about something, I feel like all of

Rinat:

those different knowledge of data centers comes together in an unorganized way.

Rinat:

In this conversation, I'm looking to gather my thoughts, organize all

Rinat:

the different knowledges I have and also learn something new, maybe so

Rinat:

thank you for tuning in and let's.

Rinat:

talk about data centers.

Amit:

Well, Thanks so much for that introduction.

Amit:

Data center is quite a relevant topic, especially during

Amit:

the age of cloud computing.

Amit:

But I think before we talk about data center, we need to actually talk about

Amit:

how People used to store data, transfer data, et cetera, before data centers.

Amit:

Companies used to manage their own IT infrastructure.

Amit:

And when I say IT infrastructure, that means all the computers that

Amit:

were provided in the offices.

Amit:

So suppose an office is a physical location at one physical location,

Amit:

you have say a hundred computers.

Amit:

You have to manage, update software make sure they're all secure,

Amit:

they're all connected and the backups are also in place.

Amit:

So that's how it all started from there.

Amit:

if you scale it up, so now you have say multiple locations and you

Amit:

still manage your computers, but each location has its own IT team.

Amit:

But what if you wanted to have a data center, say, in one city, you have

Amit:

two different office locations or five different office locations, and you want

Amit:

to manage all the IT infrastructure from a central location, and a data center can

Amit:

give you a clue about what it actually is.

Amit:

It is basically storing the data in a central location.

Amit:

That's why it's called a data center.

Rinat:

So as soon as I, think about this definition, this is what starts

Rinat:

the alarm bells in my head because one of the major movements what

Rinat:

we're trying to do is decentralize everything when it comes to technology.

Rinat:

There is the saying that don't put all your eggs in one basket.

Rinat:

From that perspective, data is one of the most important assets a company

Rinat:

nowadays can have, whether you're an it company or even in non it company

Rinat:

Putting all of your data in one place seems really counterintuitive.

Rinat:

So what is the deal here?

Amit:

I don't know the exact answer, but if I take a guess, I think it

Amit:

is because you want to manage the infrastructure in a more scalable way.

Amit:

Because if you have to manage, say 100 computers, And the data

Amit:

on those computers, then it takes a lot of time and effort.

Amit:

Plus you have to go physically to separate locations restore the hard drives manage

Amit:

the network infrastructure, et cetera.

Amit:

But if you have a data center where all the backups are stored,

Amit:

then you know that, , in case a computer fails or in case a person

Amit:

leaves or whatever, I think the infrastructure is far easier to manage.

Amit:

Once we understand what are the different types of data center, I

Amit:

think that would make a lot of sense because I think a data center is

Amit:

not applicable for say a startup.

Amit:

It could be, but in a very different manner.

Amit:

Say you have computers and your IT infrastructure, your

Amit:

server in one central location.

Amit:

And when I say a server, the server is basically hosting a webpage, which

Amit:

can be accessed over the internet.

Amit:

So you're hosting your server where all the files are stored, your

Amit:

database is there, and people in the office are able to access the

Amit:

information on the database and work on the server and create your website.

Amit:

Plus you have your local machines and these desktop machines

Amit:

are managed by the IT team.

Amit:

So that's one part.

Amit:

Now, suppose you are a very big company, you have thousands of people and you

Amit:

want to manage the data of not just the employees, but also the product.

Amit:

So suppose you're making a CRM application and you want to manage that.

Amit:

those CRM application has a lot of data coming from the customers

Amit:

and you want to store that data.

Amit:

The data center is not the data of the people in the office, but

Amit:

the data is of the people who are actually using your service.

Amit:

I think the key thing is It's in the word data center.

Amit:

It's a center of data, but there are two types of data in any organization.

Amit:

One is the data of the employees who are working at the offices and the other

Amit:

data is the data of the customers who are accessing the services of the I.

Amit:

T.

Amit:

Company.

Amit:

So suppose I want to manage my own I.

Amit:

T.

Amit:

Infrastructure.

Amit:

So for that, I don't need to manage a lot of infrastructure in terms of data center.

Amit:

But if I want to manage my customers data, then I think that has to be managed

Amit:

in a far simpler way because I need to first make sure that the customer

Amit:

is able to access my infrastructure securely, safely and very quickly.

Amit:

So they get access to that information.

Amit:

Plus they can get access to that information 24 7.

Amit:

So it means it's always on plus it has to be secure so that no

Amit:

one is able to hack into that data and leak it onto the internet.

Amit:

So that's why it makes sense to have a a data center.

Amit:

This data center could be managed by the company.

Amit:

So suppose I build a CRM application, so I manage the data center.

Amit:

I take care of the physical space in which the servers are located.

Amit:

So there are two aspects to it, right?

Amit:

You have to manage the physical space.

Amit:

So people can't break in and the computers will get very hot so you

Amit:

have to provide some cooling, you have to maintain the humidity, you

Amit:

have to provide access control.

Amit:

So no unauthorized person can access the data.

Amit:

And you have to make sure that it's compliant with all the regulatory bodies.

Amit:

And then you have the physical infrastructure where in you have

Amit:

the computers, the networking equipment and the servers.

Amit:

So that's part of a data center.

Amit:

Now, in a different data center, the physical space itself is

Amit:

managed by a separate company, but the equipment, computer equipment

Amit:

is managed by the company itself.

Amit:

So suppose you are a company and I manage the physical space.

Amit:

You say, I want to rent your space a hundred square meters and I want

Amit:

to put my network servers there.

Amit:

I'll provide the security.

Amit:

I'll provide the cooling, but you bring your computer equipment.

Amit:

So that's co location data center.

Amit:

The third one is where I give this physical space.

Amit:

And I also provide the physical infrastructure of the computing equipment.

Amit:

So I'll provide the computers, I'll provide the space and then

Amit:

you can use that as a data center.

Amit:

And the final is cloud service provider like Amazon web services.

Amit:

Google cloud or Microsoft Azure.

Amit:

And these are infrastructure that is all virtualized, which can be

Amit:

accessed from anywhere in the world.

Amit:

So these are the four major data centers.

Rinat:

Wow, that's quite insightful.

Rinat:

I actually didn't know that you can divide up different types

Rinat:

of data centers in this way.

Rinat:

It's a good thing to learn about as well.

Rinat:

While you were describing all of these, I was also thinking that what are the

Rinat:

motivation behind having each of these types and, going back to the first

Rinat:

question I had is, why are we, increasing risk by having everything centralized?

Rinat:

But then I thought it's all about isolating the problem and finding the

Rinat:

solution for that particular problem, when someone says, don't put all of

Rinat:

your eggs in one basket, they are trying to say that, okay, If you lose

Rinat:

the basket, you lose all of your eggs.

Rinat:

So how does it translate into data centers?

Rinat:

If you put all of your data in one place, and then if the physical

Rinat:

location is subject to an earthquake or a fire and you lose all of

Rinat:

your data, so that's one problem.

Rinat:

And there is a solution for that problem, which is having backup,

Rinat:

online backup, offline backup, very quickly retrievable backup.

Rinat:

And the same way there's all of these other problems that centralizing

Rinat:

data comes with, there are isolated solutions for all of those as well.

Rinat:

Ultimately, it's a addition and subtraction of what you gain from it.

Rinat:

So you save a lot of cost by centralizing it.

Rinat:

And even having two or three different backups, you would still save money by

Rinat:

having a centralized location for all of your data because it would just reduce

Rinat:

maintenance and management of it as well.

Rinat:

So you would have to hire less people to manage it all together.

Rinat:

And that's a lot of money as well.

Rinat:

It's not just the infrastructure management, but also a huge team of

Rinat:

solutions architects and data architects who would, make everything work together.

Rinat:

After all of those calculation, it's more worth it to have a centralized data

Rinat:

center, and that's why we have them.

Amit:

you say centralized data center, it doesn't mean that it's centralized

Amit:

as in it's just one data center, right?

Amit:

I think you have to look at the Cost and the growth of a company.

Amit:

So if a company is very small, you can manage one server at your own office.

Amit:

because you don't need a lot of hard disk space.

Amit:

You're not a Facebook or a Google.

Amit:

Imagine every like of a Facebook is stored.

Amit:

And if you liked a content 14 years ago, it's still stored.

Amit:

I graduated my university in 2006 and that's when Facebook came.

Amit:

It's 2024.

Amit:

So it's 18 years since I graduated.

Amit:

It has stored every single like every single photo, that I've

Amit:

clicked on every single comment.

Amit:

And there are 1 billion people accessing it.

Amit:

So imagine the amount of storage in terms of images, files, et cetera.

Amit:

Now, if you have to store that physically in one single location, of course, that

Amit:

can create a failure because if that place crashes, then all the data is lost.

Amit:

But when you create a data center, you have to look at what, The

Amit:

end goal is if I'm a very small company, I don't need a big data.

Amit:

I don't even need a data center.

Amit:

I can manage the infrastructure in my own company.

Amit:

I just need a small team to handle the maintenance of the servers to

Amit:

update the security patches to control the networking equipment and to have

Amit:

multiple hard disk in a RAID format.

Amit:

So RAID is basically to provide redundancies for hard disks.

Amit:

So basically if you have multiple hard disk, you can put them in

Amit:

different RAID configurations so that it can create automatic backups

Amit:

without you configuring it regularly.

Amit:

So that's one part.

Amit:

But suppose now you have A million customers.

Amit:

Now you can't have that server in your office because it

Amit:

will require a lot of space.

Amit:

plus you have to manage the cooling, then it's better you move to a separate

Amit:

physical location from your own office.

Amit:

So that way you can minimize the cost because now you have a location

Amit:

just for optimized for data center.

Amit:

you don't need tables.

Amit:

You don't need chairs.

Amit:

You just need a very big empty space.

Amit:

You need a good air conditioning.

Amit:

You need humidity control and you need people to guard and secure that location.

Amit:

So it's a very simple box in which you stack a lot of servers.

Amit:

If I want to now scale my infrastructure because I have that big empty space,

Amit:

I can scale it up very quickly.

Amit:

But if I have the servers in my own office, I need a different room.

Amit:

And then I have to figure out how to connect the servers into different rooms.

Amit:

But if it is just one physical location, it's just like attaching one cabinet,

Amit:

putting the hard drives, putting the network cable and then connecting it.

Amit:

And then it's all online, right?

Rinat:

Yeah, absolutely.

Rinat:

You touched on a a very important aspect of I.

Rinat:

T.

Rinat:

Entrepreneurship and for our non techie listeners out there, you

Rinat:

might think that if it's a technology company, then you should be able

Rinat:

to handle having a data center.

Rinat:

Obviously, depending on the need, the size and scale would differ uh, But, if you're

Rinat:

an IT company, it's your bread and butter.

Rinat:

And this is where I want to actually use an analogy of, a restaurant or a

Rinat:

fast food, say, a burger restaurant.

Rinat:

If it's just one branch and you're starting up, it makes more sense to

Rinat:

prepare the burger patties and all of the individual items in your kitchen

Rinat:

and then prepare and serve the gourmet burger and that's your business, but

Rinat:

as your business grows and then you compare, start comparing it with fast

Rinat:

food chains like McDonald's and the similar, where, you have grown to a degree

Rinat:

that it no longer makes sense to create the burger patties at your location.

Rinat:

You probably have a separate central location where you create the

Rinat:

burger patties for five or ten or a hundred restaurants in the area.

Rinat:

it also could come to a point where you outsource or delegate to another

Rinat:

company who are better suited to cut and slice the potatoes into fries.

Rinat:

Even though you are a fast food shop and, you don't think that just because

Rinat:

of that McDonald's should, prepare all of their food by themselves.

Rinat:

No, as you scale up the sort of the load of work becomes so high that it makes more

Rinat:

sense to delegate or give that particular part of the task to a different company

Rinat:

who would just do that and supply that.

Rinat:

Even in IT company, you could specialize in say, for example, social media being

Rinat:

a social media platform like Facebook.

Rinat:

So your expertise is not to manage data in cloud, . So you might want

Rinat:

to hire, I don't know what the actual configuration is Facebook's

Rinat:

backend, but you might want to.

Rinat:

Microsoft Azure or Amazon Web Services, who would handle all

Rinat:

of your cloud infrastructure.

Rinat:

And that way you can focus on what you do best.

Rinat:

In case of Facebook, I don't know actually what they do best.

Rinat:

But whichever IT company that we can talk about whatever their expertise is, they

Rinat:

can focus on that and they can outsource or delegate this part of managing

Rinat:

data which is a very important part.

Rinat:

But, as it scales up, it becomes a very big challenge to handle by themselves.

Rinat:

And that's where cloud data centers comes in.

Amit:

yeah, the analogy is right.

Amit:

The patties itself won't be manufactured at that single location.

Amit:

It's prepared at that location, but it's not manufactured.

Amit:

So patties are maybe prepared somewhere else.

Amit:

And then it's distributed to different locations.

Amit:

So yeah, that does make sense.

Amit:

I think with a data center, some companies can't even afford

Amit:

to have their own servers.

Amit:

So cloud service providers, provide access to a networking and

Amit:

storage solution over the internet.

Amit:

instead of buying multiple servers you basically rent it.

Amit:

And you say, I will buy a portion of your infrastructure with this processing

Amit:

power, this CPU, this storage etc.

Amit:

And I will rent it out for a certain period.

Amit:

And I will pay a subscription cost for it.

Amit:

They can provide the infrastructure plus the configuration.

Amit:

So they'll manage the firewall, they'll manage the DNS servers.

Amit:

You don't have to do the configuration.

Amit:

They will also install all the softwares.

Amit:

So you have the physical hardware, the hard disk RAM and the CPU.

Amit:

Then you have the configuration.

Amit:

You have to set up a firewall.

Amit:

You have to set up a DNS.

Amit:

You have to set up the server software and a database.

Amit:

So they do that.

Amit:

And then you have the software to manage all this, right?

Amit:

So there are different levels of what a cloud service provider can do.

Amit:

And based on your requirement, you can then say, okay, I want to pay for this

Amit:

and I will manage it in my own way.

Amit:

I'll configure it in my own way, but I'll rent it from them.

Amit:

And because I'm renting it from them, I don't have to worry

Amit:

about the physical security.

Amit:

I don't worry about cooling.

Amit:

I don't have to worry about paying the electricity bill.

Amit:

I don't have to worry about securing the physical location with say security

Amit:

staff, and then I don't have to worry about updating the infrastructure

Amit:

because I'm now getting the latest CPUs and the GPUs, which I need.

Amit:

So that's a benefit of a cloud service provider.

Amit:

Of course, as I mentioned, you have managed service providers, co location and

Amit:

companies having their own data center.

Amit:

So sometimes when you are such a big company, it makes sense

Amit:

to have your own data center.

Amit:

Amazon or Microsoft may be having their own data centers.

Rinat:

Microsoft are the best at one of the best in the market.

Rinat:

So they would have their own, which is intranet and contained within

Rinat:

their company, which manages or maintains all of the data inside.

Amit:

need to give you an example, like the first cloud computing platform started

Amit:

with AWS, Amazon Web Services, right?

Amit:

But before that, what did we have?

Amit:

We had virtual machines.

Amit:

So before that, what we used to have is we used to virtualize the hardware

Amit:

itself, virtualize on the hardware.

Amit:

So suppose you have one single machine with is very high capacity.

Amit:

What do you try to do is you try to say that, okay, I

Amit:

have this one single machine.

Amit:

Now I will install 10 different virtual machines on this.

Amit:

And this single hardware can serve 10 different people.

Amit:

So that's the virtual machine.

Amit:

Now, when Amazon became such a big company, it had to create a big data

Amit:

center to manage all the shipping orders and the logistics and the

Amit:

data that is actually created to create the whole website, right?

Amit:

So it has to manage all the images, it has to manage all the pricing, it has to

Amit:

manage all the authentication, login, etc.

Amit:

And plus store all the orders.

Amit:

And when it started growing big it created big data center.

Amit:

And then it thought, okay, I am managing all this data center and

Amit:

it's not getting fully utilized.

Amit:

Maybe I can rent a bit of it to other people.

Amit:

That's how cloud service started.

Rinat:

Yeah, that's such an interesting way of new product

Rinat:

coming into market, right?

Rinat:

A lot of tech entrepreneurs have built their whole business.

Rinat:

They were initially doing something else.

Rinat:

But in order to support that something else, they created even a better product.

Rinat:

And I think some of our guests in the past were also talking

Rinat:

about, similar kind of things.

Rinat:

we had the Orkes for Netflix where they were trying to create something.

Rinat:

And then.

Rinat:

They ended up creating something which was really, necessary for a lot of people.

Rinat:

So they ended up selling that.

Rinat:

it's very interesting how you come up with new products and there

Rinat:

are so many more opportunities coming every month, every year.

Rinat:

People might think that.

Rinat:

I.

Rinat:

T.

Rinat:

Has oversaturated with all of these new businesses, . But there is new

Rinat:

inventions coming on every few days.

Rinat:

And right now we're seeing a boom, with A.

Rinat:

I.

Rinat:

And new products will be made.

Rinat:

Cloud services and all of these things.

Rinat:

All of these were possible as the Internet became, more commonplace.

Rinat:

That's why you could think of, okay, I have broadband, I have MBPS, not

Rinat:

KBPS, so I could stream data, I could, collaborate online with multiple people.

Rinat:

All of this became possible as the Internet speed grew.

Rinat:

So now with AI and large language models and content generating

Rinat:

models, which generates video, pictures, audio, all of these things.

Rinat:

Now we're going to see, very soon , a new type of products that are emerging

Rinat:

that people didn't even think about.

Rinat:

It's an exciting place to watch the space and also think about yourself.

Rinat:

Do something, maybe the thing that you set out to do that, that might not be

Rinat:

the thing that makes you successful.

Rinat:

But, in order to make that possible, you'll probably

Rinat:

build something else around it.

Rinat:

Which will then become the thing that makes you known and successful.

Rinat:

I think one of the things that I always encourage everyone

Rinat:

is to basically do something.

Rinat:

implement something, take action on anything and you

Rinat:

never know how it will turn out.

Amit:

This problem is faced by so many other people.

Amit:

So you can create a business out of it and monetize it.

Amit:

Yeah.

Amit:

I think you're absolutely right.

Amit:

We have talked about data center, but let's talk about the challenges of

Amit:

the data center and maybe what the future lies ahead for data centers.

Amit:

So one of the biggest challenge for opening any data center is not the

Amit:

hardware, not buying the software or the network switches, routers, etc.

Amit:

It's basically the energy.

Amit:

They consume a lot of energy.

Amit:

So you need to get subsidized electricity.

Amit:

Or subsidized energy in a form that is cheap for you to operate it plus because

Amit:

they consume a lot of energy to operate, they also generate a lot of heat.

Amit:

So you need a lot of cooling.

Amit:

So actually, data centers main cost comes from these particular things.

Amit:

So you have energy and you have the cooling and then you have to

Amit:

maintain the humidity because you don't want any computer infrastructure

Amit:

to get spoiled because of humidity.

Amit:

So you have to manage the energy, manage the humidity, manage the cooling.

Amit:

So once you've managed all that, you need to manage the physical

Amit:

location with security personnel, provide access control, et cetera.

Amit:

Plus you have to update the hardware because hardwares do fail because

Amit:

they are constantly running at a very extreme temperature in a very sensitive

Amit:

environment that in case one fails, you have to immediately replace them.

Amit:

Now, of course, all these servers have their own backups.

Amit:

In the same location or in different location in a different data centers.

Amit:

So if you think of AWS or Google, they don't have one physical location

Amit:

where all the data is stored.

Amit:

They have multiple data centers across the world.

Amit:

So they have one data center in say UK, one in Europe,

Amit:

one in US, one in Australia.

Amit:

So suppose you are based in Europe.

Amit:

They're going to serve the data from Europe servers, not from America,

Amit:

because the latency will be very high.

Amit:

That means that the time it takes for the data to travel

Amit:

from US to Europe is quite high.

Amit:

So they would rather serve it from a location closer to you so that

Amit:

They can instantly serve you the data, plus it creates a backup.

Amit:

So suppose there is a tsunami in Australia, then the data stored in

Amit:

Australia is also backed up in, say, U.

Amit:

S.

Amit:

and Europe.

Amit:

So not all the data is lost immediately because they have redundancies in place

Amit:

in terms of different physical location.

Amit:

So that's one part.

Amit:

So you have to manage the heating and to generate the energy The

Amit:

couple of option is you can go for generating your own energy.

Amit:

You can have solar cells but the solar panels will not

Amit:

generate enough electricity.

Amit:

So you have to maybe burn fossil fuels or you have to go for a nuclear power

Amit:

plant option, et cetera, et cetera.

Amit:

So it depends on how sustainable you want to go.

Amit:

And a lot of companies now they buy carbon credits and they say that,

Amit:

okay, our data center is completely carbon neutral because whatever,

Amit:

emission they produce, they are neutralizing it through some other means.

Rinat:

No, I was just going to add that it feels like quite an unethical thing to do.

Rinat:

That's not the purpose of these things that are offered.

Rinat:

Rather than actually working to reduce your carbon footprint,

Rinat:

you're just buying it off.

Amit:

Think of it like this.

Amit:

Your main job is to provide IT services to multiple people or

Amit:

billions of people in the world.

Amit:

Your job is not to manage the carbon emissions.

Amit:

Someone else is managing that carbon emission for you, and they are saying

Amit:

that if you produce this much emission, we will plant this much trees so we

Amit:

will take care of that on your behalf.

Rinat:

Okay.

Rinat:

I didn't realize that this is how it works.

Rinat:

Of course, in that case, yeah, this is just another way of, doing the same thing.

Rinat:

You're outsourcing

Amit:

Because that's not your main expertise, right?

Rinat:

That's actually good to know and understand.

Amit:

So once you have all this infrastructure, now you have to manage it.

Amit:

So someone will come, they will service the hard drives, they will

Amit:

update the softwares, they will, if there is any hard disk fail, or they

Amit:

have to retrieve some information because government has asked for it.

Amit:

So then there are different aspects to that as well.

Amit:

And then humans will have to enter the environment.

Amit:

So they'll have to make sure that They retrieve the right hard disk

Amit:

to retrieve the right information.

Amit:

We can replace that with robots so they can retrieve the data

Amit:

for you instead of humans.

Amit:

So that's one way where the data centers are heading.

Amit:

The other way is Using AI and machine learning.

Amit:

So you can use the algorithms to predict when the next hard disk would fail

Amit:

or when the next memory would fail.

Amit:

And you can based on that prediction, you can order the spare parts and be prepared

Amit:

to replace it as soon as it fails.

Amit:

So you have less downtime and you have instant connectivity.

Amit:

Then quantum computing is now being built.

Amit:

So how do you create a data center that has quantum computing

Amit:

provided to all your customers.

Amit:

So you have to think at that level.

Amit:

At a high level These are the challenges.

Amit:

And these are, this is the future in which the data center is heading.

Amit:

But I think one of the main things that you might have noticed, especially with

Amit:

apple intelligence getting launched recently the focus is AI running

Amit:

locally on your own hardware, right?

Amit:

So the data has to stay on your hardware.

Amit:

The algorithms have to run on your hardware.

Amit:

So in an age where data itself Is being managed on your edge device,

Amit:

which is called very close to your location or basically in your hand

Amit:

edge device could be your mobile phone.

Amit:

It could be a thermostat.

Amit:

It could be a doorbell.

Amit:

So these are edge devices, right?

Amit:

They're all connected to the internet, but they are actually

Amit:

closer to you than a data center.

Amit:

And the data is normally processed on board that computer and then

Amit:

stored at a different location, but it's processed there.

Amit:

So what happens to those edge devices?

Amit:

And because there are so many edge devices now or you can say internet of things,

Amit:

IOT devices, then where does that data go?

Amit:

You may not want to store all the information or all

Amit:

the logs in a data center.

Amit:

You might want to store it in your own house.

Amit:

So you might have your own backups or systems in place to do that, right?

Amit:

That information you don't want data center.

Rinat:

Absolutely.

Rinat:

To implement AI decision making you need to have localized

Rinat:

processing of all of those data.

Rinat:

You don't want it to send it to a central location and be processed and then come

Rinat:

back because that might be too slow.

Rinat:

for this kind of IOT devices, you want it instantaneous as if you're talking

Rinat:

to a robot helper in the kitchen you definitely want localized processing.

Rinat:

So for that how would that be managed?

Rinat:

That's whether, moving from Mbps to Gbps internet speed is a better

Rinat:

option than localized processing.

Rinat:

Because if you want to do localized, You have to think about the cost of

Rinat:

hardware that needs to be present and that's the whole point of I.

Rinat:

T.

Rinat:

and, making everything remote that, make everything more efficient by having one

Rinat:

thing in a central location, which does, which serves like 20 people or 20 devices.

Rinat:

But if you need to do localized processing, that's

Rinat:

a totally different story.

Rinat:

Obviously it could go both way.

Rinat:

We never know what the future holds.

Rinat:

But, as you're saying that one of the options for data

Rinat:

center is to move that way.

Rinat:

I'm thinking that maybe it's more the kind of the overlords of the I.

Rinat:

T.

Rinat:

It might actually want to focus more on how to improve that communication so they

Rinat:

can still do the processing remotely.

Rinat:

The data transfer is so high speed that it feels instantaneous.

Amit:

I think The cloud data centers will still exist.

Amit:

What will happen is that a lot of processing will happen on board.

Amit:

So currently you fetch some information.

Amit:

you log into Amazon, you buy a product, you buy a laptop.

Amit:

All that information will be still stored on Amazon servers and you

Amit:

can retrieve the information from your mobile app from your laptop.

Amit:

But say you have and AI algorithm running on your smartphone and

Amit:

you want to edit some images that are stored on your smartphone.

Amit:

They are not stored on the, say Google drive or Google photos.

Amit:

I'm talking about an Android phone.

Amit:

So, that processing is local.

Amit:

Plus you have certain emails.

Amit:

So My emails are stored in a data center, but the AI

Amit:

processing is happening locally.

Amit:

So now there are two aspects to it.

Amit:

The data itself that is being retrieved from a data center and the processing.

Amit:

I don't want the processing to happen in a remote location.

Amit:

I want the processing to happen in this location.

Amit:

Why?

Amit:

Because the moment the AI algorithm works on my personal data, it

Amit:

trains itself and I don't want it to train on my personal data.

Amit:

I want it to be trained on public information, and this

Amit:

is private information, right?

Rinat:

Yeah I know.

Rinat:

This is obviously a very important thing about privacy and, everyone

Rinat:

should have the right to their own data and how they're used and everything,

Rinat:

but at the same time, I also see the, Difficulty it brings in terms of I.

Rinat:

T.

Rinat:

And technological innovation, because when A.

Rinat:

I.

Rinat:

Is being trained is anonymized data, and once it learns the pattern, it has

Rinat:

no understanding of individual personal data, but it has a overall whole

Rinat:

understanding of all different kinds of email, all different kinds of people sent.

Amit:

but if you think about anonymized data, suppose you are

Amit:

writing an email to a company, right?

Amit:

And your personal company, and you will have names, you will sometimes

Amit:

have date of births, you will have some sensitive information in it,

Amit:

say banks have sensitive information, financial transactions, etc.

Amit:

And suppose you train and you use a AI in that environment, and

Amit:

now that AI model has learned.

Amit:

About all those things.

Amit:

And now someone who knows that this is the training data it has

Amit:

been trained on, you can actually figure out a way to retrieve that

Amit:

information from the algorithm, right?

Amit:

So you don't want that.

Amit:

So that's why when you talk to chat GPT, you don't say

Amit:

that, okay, this is my company.

Amit:

I work at say Amplify and I do this and this is my current problem.

Amit:

This is my boss name.

Amit:

I want to create an email.

Amit:

No, you don't want that information because that sensitive information,

Amit:

how do you think is getting anonymized?

Amit:

I don't think that if there are billions of people putting inputting

Amit:

all that information, do you think there are billions of people

Amit:

trying to anonymize the data?

Amit:

I don't think so.

Rinat:

No, absolutely not.

Rinat:

That's actually a very good point.

Rinat:

No, if those kind of information are stored then definitely, I

Rinat:

was thinking about when you said localized processing for that, you

Rinat:

need a processor powerful enough.

Rinat:

If we're talking about IOT devices, then, adding each of these device

Rinat:

with powerful enough processor to run AI algorithms is quite a costly

Amit:

So I think a lot of the mobile companies, they are now having different

Amit:

cores because I've been recently looking at thinking of upgrading my own laptop.

Amit:

So I was looking at different options.

Amit:

So they have different silicons and these silicons now have different cores.

Amit:

So some cores are just for cores which are optimized for computing tasks.

Amit:

And then there are some cores that are optimized for A.

Amit:

I.

Amit:

Tasks and a task means multiple threads, multiple processing,

Amit:

neural networks, etcetera.

Amit:

So they're optimized just for those types of tasks.

Amit:

And these silicons are now being built and they're getting installed

Amit:

on the smartphones or the laptops.

Amit:

So even in Intel series, you have the Intel Core series and

Amit:

the Intel core ultra series.

Amit:

The ultra series has AI cores in it and then the latest M1 chips from

Amit:

Apple, they have those chips as well.

Amit:

So the Apple intelligence won't work on say anything below Apple iPhone

Amit:

15 or any Mac book that doesn't have an M1 chip or a lower chip, right?

Amit:

So it will be limited by the hardware infrastructure.

Amit:

But over a period of time, that infrastructure cost will come down

Amit:

and it will become more accessible.

Amit:

And then more and more computation will be done on your devices.

Amit:

Even Google for their pixel devices, they manufacture their

Amit:

own tensor processing units.

Amit:

So I think there is definitely a possibility of Scaling it

Rinat:

Yeah.

Rinat:

Future is exciting.

Rinat:

There's so many groundbreaking innovations happened in the

Rinat:

last one or two years with AI.

Rinat:

And some of the things that are in the horizon, like quantum computing,

Rinat:

which will turn everything upside down.

Rinat:

And then obviously people, countries and companies are working

Rinat:

towards getting AGI as well.

Rinat:

And it's a very Exciting place to know about everything that's

Rinat:

happening, and that's why our podcast here, I think, touches on all of

Rinat:

these the relevant topics of time.

Rinat:

And I hope our audience would continue listening to different topics that

Rinat:

we talk about every few weeks.

Rinat:

One of the things I have heard at one point is that Microsoft is thinking

Rinat:

of putting their data centers on the seabed because it's so cold, it

Rinat:

would just be naturally easier to

Amit:

or in Arctic locations or Antarctic locations where it's,

Amit:

but there we have permafrost.

Amit:

But then of course you have to bear in mind, how do you access it physically?

Amit:

Do you station staff there physically?

Amit:

What if the polar ice caps melt?

Amit:

What if there is a leak in the underwater data center, who fixes it?

Amit:

So then there are different kinds of challenges, but yes, of course, it

Amit:

solves one part of the problem, but then it creates other problems as well.

Rinat:

Absolutely.

Rinat:

That is the story of all things.

Amit:

The main thing is that we have to now start thinking about

Amit:

space infrastructure as well.

Amit:

So once we start going to moon and Mars, we will then have different types of

Amit:

challenges because the data can't travel from moon to earth instantly, right?

Amit:

So there is a time lag.

Amit:

So you have to store it somewhere.

Amit:

Then you have to transmit it somewhere and then you have to provide good Connection.

Amit:

So how do you manage data centers, say, in a world where you have moon bases

Amit:

or your bases in Mars, so that will provide a different set of challenges.

Amit:

Now you have to provide communication between Earth, Moon, Mars, et cetera,

Rinat:

Yeah, the universal speed limit has just become too slow for us humans.

Rinat:

Now, even just 100 years ago, we were just about measuring the speed of light.

Amit:

Now we're hitting this limit off the thing, but I think I was reading a

Amit:

blog recently where they were talking about how do you operate infrastructure

Amit:

in a very slow internet connection, right?

Amit:

What it means is suppose you don't have 100 megabyte connection, which we are.

Amit:

so used to right now, right?

Amit:

So what if you operate it at 10 mbps connection or what if you are

Amit:

operating in a One mbps connection or even 500 kbps connection.

Amit:

So that happens in Antarctic because they get internet only for a few hours a

Amit:

day when the satellite passes over them.

Amit:

And that's how they get the internet connection.

Amit:

And now it's very slow.

Amit:

So in that slow internet speed, what infrastructure can you operate and how

Amit:

can you manage that effect efficiently?

Amit:

I think that is another type of challenge, right?

Amit:

It's because we're so used to high speed internet.

Amit:

We think all problems can be solved with providing more bandwidth, etc.

Amit:

But, there are locations where you just cannot provide high speed.

Amit:

How do you manage infrastructure and how do you set expectations

Amit:

for that kind of environment?

Amit:

And I think that is again a beautiful challenge to solve.

Rinat:

let's touch base in terms of data center for our non techie folks out there.

Rinat:

I hope you guys already learned a lot.

Rinat:

I have.

Rinat:

Thank you.

Rinat:

Amit for all of these insights on data center and more.

Rinat:

But just to bring it all in one one thought.

Rinat:

Data center is in one or multiple physical location.

Rinat:

We might think that it's virtual, but it's not.

Rinat:

The same way when we talked about cloud in one of our previous episodes, we

Rinat:

talked about cloud computing, cloud infrastructure, even though the name makes

Rinat:

it sound like it's somewhere over there in the sky that we don't have to know

Rinat:

about, or even it's, it may be virtual.

Rinat:

But actually, It's quite closely related in this way that , even though we say

Rinat:

it's in the cloud, it's probably in a data center somewhere, which is a

Rinat:

physical location, heavily guarded, heavily, protected and environmentally

Rinat:

controlled in a granular way to make sure that everything's functioning.

Rinat:

this physical location could be anywhere as we've discussed today under the

Rinat:

in arctics or in, in countries where electricity is probably cheaper.

Rinat:

Many things to consider when you are thinking of data center, but

Rinat:

the necessity of it is definitely something that is unavoidable for

Rinat:

small companies and for large I.

Rinat:

T.

Rinat:

Organizations I.

Rinat:

T.

Rinat:

Or not even for banks.

Rinat:

They have their own data centers a lot of the times for the very reason of security

Rinat:

and privacy off of their customers.

Rinat:

Whether you're an I.

Rinat:

T.

Rinat:

Company or not, as long as you're, a mid sized company in any industry,

Rinat:

you would probably need data center.

Rinat:

to be able to operate and provide services to your customer, and that's

Rinat:

why data center is a very crucial bit of technology that is everywhere and is

Rinat:

needed by everyone as anyone is using some sort of technology on a day to day life.

Rinat:

Anyway, so it's a very sort of important topic to know about, learn about,

Rinat:

and also to train yourself about, because as people are talking about,

Rinat:

AI taking over people's jobs and which skills are becoming out of date, etc.

Rinat:

Anything to do with data centers as it may evolve.

Rinat:

The technology, but knowing about data center being trained and certified on

Rinat:

various aspects of data center will always be in place because, AI or any other

Rinat:

technology can't survive without data and it being transmitted very quickly.

Rinat:

So that's why it's very important.

Amit:

I think one of the things that we are now seeing is that AI

Amit:

needs a lot of processing power.

Amit:

Most of the servers were built using CPUs, not GPUs.

Amit:

So now there are new data centers being built with GPUs.

Amit:

And the problem with GPUs is they consume far more energy than a CPU.

Amit:

So if you have a lot of GPUs here, hundreds and thousands

Amit:

of GPUs in one place.

Amit:

One physical location, it sometimes requires far more energy than is

Amit:

actually consumed in one city.

Amit:

It requires a very large amount of electricity and that electricity can be

Amit:

provided through maybe hydroelectric, nuclear, fossil fuels, et cetera.

Amit:

So it creates its own challenges now, especially with the infrastructure because

Amit:

you have to train these large language models with tons and tons of data.

Rinat:

We've talked about a lot of challenges around data center, and

Rinat:

those are quite serious challenges that needs to be solved, even if

Rinat:

we have a solution right now, there needs to be a better solution.

Rinat:

So to our audience, can you guys think of any of these challenges

Rinat:

and what could be the solution?

Rinat:

It doesn't have to be really, detailed tech solution, but whatever you can think

Rinat:

of, reach out, tell us what you think, what the solution could be, and we could

Rinat:

be bounce ideas off of each other.

Rinat:

Tell us what your idea is.

Rinat:

And we could say, okay, this seems plausible.

Rinat:

You should, pursue it more.

Rinat:

So it would be an exciting thing to do.

Rinat:

we look forward to getting your feedback and communication on how we would

Rinat:

solve a lot of these challenges that we talked about today Thank you very much.

Rinat:

Very much enjoyed talking about data centers.

Rinat:

Hope our audience did the same.

Rinat:

We look forward to getting your feedback and to hear from you.

Rinat:

with that thought I'll end here.

Rinat:

Thank you everyone for listening.

Amit:

Thanks everyone.

Amit:

Bye.

Rinat:

Bye.

Next Episode All Episodes Previous Episode

Listen for free

Show artwork for Tech Talk with Amit & Rinat

About the Podcast

Tech Talk with Amit & Rinat
Talks about technical topics for non-technical people
The world of technology is fascinating! But it's not accessible to a lot of people.

In this podcast, Amit Sarkar & Rinat Malik talk about the various technologies, their features, practical applications and a lot more.

Please follow us to hear about a popular or upcoming technology every week.

#Tech #Technology #Podcast

Find us at
Amit Sarkar - https://linktr.ee/amit.sarkar007
Rinat Malik - https://linktr.ee/rinat.malik

Contact us at - https://forms.gle/AauF6eic2CQv2Lvn9

Review us at - https://www.podchaser.com/podcasts/tech-talk-with-amit-rinat-1556283

About your hosts

Amit Sarkar

Profile picture for Amit Sarkar
Amit Sarkar is an experienced software professional with over 15 years of industry experience in technology and consulting across telecom, security, transportation, executive search, digital media, customs, government, and retail sectors. He loves open-source
technologies and is a keen user.

Passionate about systems thinking and helping others in learning technology. He believes in learning concepts over tools and collaborating with people over managing them.

In his free time, he co-hosts this podcast on technology, writes a weekly newsletter and learns about various aspects of software testing.

Rinat Malik

Profile picture for Rinat Malik
Rinat Malik has been in the automation and digital transformation industry for most of his career.

Starting as a mechanical engineer, he quickly found his true passion in automation and implementation of most advanced technologies into places where they can be utilized the most. He started with automating engineering design processes and moved onto Robotic Process Automation and Artificial Intelligence.

He has implemented digital transformation through robotics in various global organisations. His experience is built by working at some of the demanding industries – starting with Finance industry and moving onto Human Resources, Legal sector, Government sector, Energy sector and Automotive sector. He is a seasoned professional in Robotic Process Automation along with a vested interest in Artificial Intelligence, Machine Learning and use of Big Data.

He is also an author of a published book titled “Guide to Building a Scalable RPA CoE”