Data Visualization

Rinat Malik: 00:00:00

Hi, everyone. Welcome to Tech Talk a podcast where Amit and I talk about all things tech. Today we're going to talk about a topic called data visualization. There are many nuances of data and data related topics. And we want to talk about them all to be honest, and it's quite an interesting area altogether, but I thought we could tackle one by one and one of them I am quite excited about or I think quite relevant in nowadays. With all the tech innovation that's going on is data visualization, how we visualize the data that are there right now. Gathering data is not a problem at all because there are so many machines that gathers not only personal data, but a very, a lot of like natural data. There is no lack of data, but how we visualize this data in a way that is meaningful and we can make data driven decisions that's actually become quite relevant and important. So, I thought the audience would be quite benefited from this this topic if we talk about it and share our two cents and also would be looking forward to knowing what you guys think of what we talked about today.

Amit Sarkar: 00:01:16

So, thanks, thanks Rinat for the introduction. I think your data visualization is quite important. While you were just doing the introduction, I realized that yes, we have a vast stream of data like if you go to Facebook, Facebook is collecting a lot of information. So, is WhatsApp, Instagram, Snapchat, Reddit, and many other apps and what information they're collecting is what are you liking? What are you who's liking what, what's the what kind of posts you like, what kind of posts you share? What what's the time of the day you're active. And then who whom are your friends and how frequently do you go on to which friends’ profile so then what kind of videos do you watch and how long is the normally the video length? How long do you watch it for? So those kinds of information that's just one person. Now imagine I come from a demographic of the 30 30 30+ male from Asia, who's based in London. And now that's a demographic so that data is also collected. And with all this data, and that's just one person. And imagine now you have data from 1 billion people because that's the number of users on Facebook. So then what do you do? And you want to analyze and you want to predict so you have the data, you want to make some predictions, but you then want to visualize how the pattern is, what kind of patterns are there and that's where the visualization comes because you can clearly see patterns quickly when you try to visualize them. If you see just data raw data, you won't be able to visualize them quickly.

Rinat Malik: 00:02:46

Yes, yes, absolutely. So, let's tackle this topic with a zoomed-out view at first. So, we have data. So, data has a journey it travels through various steps. So, if we start with this journey, the data is first created by the user or naturally, you know, for example, weather data is collected from natural occurrences etc. So, data is sort of created and then it is collected and then it is stored. And then we have all of these data. What do we do I mean, what is the point of gathering all the data we want to make data driven decisions all the decisions making should be based on some sort of statistics, some sort of reasoning or justification and the data is the best way to justify a decision because we can correlate to different events and see that there is a relation and then we can predict. We change one aspect then the other aspect might predictably change. So that's the whole benefit of dealing with data. That's what we can get out of data. Now. We have had many technological innovations so we to collect and store data. I mean, you know, collecting data as Amit you mentioned, Facebook and you know, so many other sort of internet, social media platforms and other internet websites and mobile apps, etc. And that's just one side of things. But you know, even your Internet of Things, for example, all the devices around you, as well as data is being collected with, you know, from natural sources, you know, weather data and then satellites. And everything is collecting and storing data. Now, what do we do with so much data that is incomplete, incomprehensible for a human to deal with and even in space, as well, you know, James Webb telescope and all of this data is being collected. We need a way to make a judgment based on data make a decision based on the correlation and causation that we find out of those data. Now, how do we do that meaningfully is where Data Visualization Step comes. Forward. We need to visualize this data in a way that is meaningful to us humans, as well as also potentially AI training modules who can learn from the patterns of these data. So, the data has two uses. One is to training an AI module and the other one is for humans to visualize in a way that we could make meaningful decisions. Now, how do we make meaningful decisions we need to have enough information in a data visualization sort of output. So, it gives us the right correlation and causation and this is actually very important sub topic in terms of data to me, correlation and causation of data. So, there is a statistical fact really didn't want to call it that. When it's sunny, the sale of ice cream goes up. And also, the sale of sorry also the reported assault cases goes up. So, does it mean that you know, there is a really no causation between ice cream consumption and assault? Now, it's because they are correlated, but they're not. They don't have a causal relationship. One doesn't cause the other. For example, there is another one the number of people who are Nobel Prize winners. The majority of them likes dark chocolate, does it mean that if you now start eating dark chocolate, you will certainly you'll have more chances of winning the Nobel Prize? No, these are correlated data, but not, you know, causally related. So that's why we need to distinguish how we're visualizing data in a way that we can make the right decision to stop or to reduce a salt there is no point reducing the sale of ice cream. Because, you know, it was after more research it was found out that it's because it's sunny and in hot weather people are more hot headed. So, two, individually unrelated statistics could be wrongfully causal, correlated. So that's why it's very important that to know how we are visualizing the data, whether it's actually giving us a meaningful insight on the topic at hand. So that's, that's, that's an interesting side of data visualization. Now we can talk about all the media that we consume the data or we visualize the data and you know screen is one in our computer monitor we visualize and in different models like pie charts, bar graphs, etc. But nowadays with technological advancement, we have like 3d visualization of data. Usually for four decades, we've had x and y plot and C, but now it has become quite commonplace to see the data in a 3d sort of visualization with X, Y and Z. So, we can sort of rotate that that plot and see how three axes are related to each other and correlated with each other. So, what are the some of the other ways we can visualize data? Amit?

Amit Sarkar: 00:08:35

Well, I think that's been first thing that pops up in my head is that the standard line, I mean, the Indian School we used to use these graphs right for drawing the equation so you have x plus y equal to some number, and then you put a value for x and then you get calculated the value for y. And then you plot those points and then you try to connect and then you get a graph. So that's the first thing that you know. And then when you start using Excel, then you get these lines line graphs, bar graphs, pie charts, etc. So that's one way to visualize. But then there was a, there's a, I forget the name of the author. But he wrote a very good book about data visualization, and you can, it's not just about visualizing data in x and y axis, or even Zed axis, because that's just one way to look at data because you can imagine data in terms of population. So, like, how do you imagine data in terms of population sizes? So, suppose you draw the population of say, India, China and all the countries and then you individually draw a circle which indicates the size of the population. Then quickly, you can see that the circles for China and India are quite big. Well, the circles for other countries are very small. So that gives you an idea of the size as well as the proportion in which they are different. So that's one way to visualize the data where there is no x and y axis and there are many other ways like you have the you have the sub Word, Word, a heat map or something, where you see based on the feedback of a particular event, like you see what kind of words were used or what kind of feedback was given and then you calculate Okay, the bigger words are again bigger in size and the small and words that are used less frequently are smaller in size. So that again gives you a visual representation which is quite different. Then you have these infographics which are now quite popular, which gives you a lot of other information. So, it's, it's like data, but then data visualized in a completely different way. Like, you're not just adding just the numbers and the graphs, but you're now adding information like okay, I can add animation, I can add emojis, I can add images, I can even add videos and that's one way to look at data and I think that's quite powerful. And it's becoming more relevant because as you said, technology has advanced and things have become more better in terms of like, how we create takes a lot of people are becoming familiar with these tools. So, it's again, very easy. I mean, we normally when we collect the data for our podcast, we visualize it in couple of different ways. We look at the map of the world, and then we look at where the audience are. So, we look at different percentages. Then you have weather maps, weather maps, again, it's a data visualization. So, it shows temperatures like across the across the UK, which parts are high, which parts are hotter, which parts are cooler, and then you have a weather maps etc. So, this whole new way of visualizing data, but in the end, it presents an information on which you can take a decision. So, if you look at a weather map, you can then decide okay, these parts of the UK are quite hot, so I might need a air conditioner next year. These parts are cool, so I don't need air conditioner, but I still need to be careful. So, things like that.

Rinat Malik: 00:11:55

Yes, absolutely. I really liked the way you've talked about different ways of visualizing apart from just xy plot or even x y Zed plot like you know there's color change heat map then there is the, you know, the circular representation of population and all of that. Then there is another word. One I find quite interesting is word cloud.

Amit Sarkar: 00:12:15

Yah word cloud. That's what I made sorry. I didn't get the right word.

Rinat Malik: 00:12:22

Yes, absolutely. Word Cloud. I think it's so catchy, right? I mean, you don't even have to think very well I mean, the one the words that are that have been used the most are bigger in size. So, you can just easily see them and their variation of word clouds as well. So, you know, for example, the stock X in stock exchange the stocks that are most volatile or most moving could become bigger, or even in IT support scenario, you know, the IT tools which are most problematic for a support team. They could just see that those becoming red and on top of the list are blinking and giving that visualize the data to make a decision or taking action very quickly and identifying I mean, you know, identifying where the problem is with good visualization. So visualization I think is a very important step of the journey of usability of data. I mean, you know, with technical innovation we've you know, conquered all of these aspects, collecting, storing, but visualization. I think there is already a lot of innovation but there is still a lot more room for more and more innovation. Because, you know, that's the crucial point where humans are making meaningful decisions and if those decisions are sort of misled by wrongly visualizing a dataset that is very dangerous in a lot of cases. And I want to sort of, I don't want to say enlightened but I want to sort of make our audience aware of some of the sort of the nuances of data visualization and how it could be dangerous for example, one of the things that a lot of controversially may be a lot of politicians and a lot of sort of biased, you know, commercial advertisements do is give you data that looks you know, a lot more believable to you as a consumer, for example, a lot of the shampoo and cosmetics app ads and this is something I noticed quite you know more very often and I'm not going to name any names of any brand, but a lot of the times they say 90% of, you know, woman or people use the shampoo and they said it was positive. And then in the end there is an asterisk and in very small letters, it is Rachel that this study has been done on 56 woman or 24 woman or 24 people, etc. So that's a very small cross section of people among the whole of the population to you know, get a deciding amount of data or enough like sort of saturation of data to make a decision 90% of 20 people is is not, you know an accurate representation of whether that shampoo or that stuff works. You know, sometimes 70 People 90 People but it's usually always less than 100 people who on who the This experiment has been done and you know, this this does not represent the population of a country or the world or any demographic, to be fair. That's one way and the other way a lot of the times various businesses or politicians I've seen sort of tried to sort of give you a biased information is by giving you an XY plot, but on the x axis, they don't tell you the unit. So the number of cars sold, you know, by a car company they could you know, compare it against their competitor car company, let's say for argument's sake for them to OTA and if it is, you know, if they're comparing they're not, you know, the start point the x axis might not start from zero. So, if you don't start from zero, if you start from say 10 million and show the comparison between 10 and 12 million, and the difference, the difference would look a lot bigger than the actual difference. So say for example, if Ford is giving an advertisement and saying that we've sold more cars than Toyota and we we've sold 11 and a half million, and Toyota sold 11 million, but if the scale is from 10 to 12 It would seem like there is a massive difference between Toyota and Ford car sales and the people might think that all four car Ford cars are more popular, so I might want to buy some, but actually the difference is actually probably 5% if you if you count from zero, so these are the things that we want to be mindful and careful of when we are visualizing data, that whether it's meaningful and whether it's the whether it's biased or not. So, x axis and y axis should always start from zero and should always show the unit of measurement. I mean, you know, sometimes, you know, you're familiar with maybe if you are based in the US where you're familiar with dollar, but they're doing a comparison of a different currency or a different length unit, and that might actually give you a skewed impression after data. So, these are some of the things that are always to be mindful of that what are they showing in x and y axis? Where is it starting from, whether it's starting from zero or not? And what are the individual sections? You know, how big are they? So, these are some of the things that I would I would ask the audience to be mindful of whenever they're visualizing a data set.

Amit Sarkar: 00:18:14

Well, I think those are really good points because I also thought like, oh, what will you talk about when it comes to nuances, but those are really good points because a lot of times there are these surveys in the newspapers and surveys on the websites that you read. And most of the surveys they're done about 1000 people 2000 people, and they represent the and they say that okay, 80% of people will have cancer, if they have meat or 80% of this will have that. But it is not a representation of the whole population. It just generally if in that mean people when they do these surveys, whoever is conducting the surveys, they think that if they select a good group, which represents most of the population, then it would you can extrapolate it to the whole population and it would give the same designs but actually that's not the case. And that's where you have to be very careful. And the same goes with polls as well. So, when an election is going on when the poll results come or 80% So like Trump and Hillary Clinton, so everyone says, oh, Hillary Clinton will when Hillary Clinton will win and then Trump won the election and same happened with Brexit, or Brexit won't happen. Brexit would happen and then Brexit happened. So, everyone's shocked and everyone was surprised because the polls told a different reason, because you're talking to only people who are willing to talk to you.

Rinat Malik: 00:19:32

Yes, absolutely

Amit Sarkar: 00:19:33

again, that’s Part of the whole survey process.

Rinat Malik: 00:19:37

And that's actually very important thing you mentioned and this and they actually want to also mention, I don't know how related it is to data visualization, but one of the things that I feel is a lot of the things a lot of the content we consume in social media, you know, especially in Tik Tok, and nowadays reels and shops, you know, many, many sorts of sort of short videos that we consume. And we think, okay, these are, these are a representation of all the people what people thinks and et cetera, et cetera. But actually, these are only the people who are extroverted enough to sort of pick up the camera and take a video and share their opinion and, you know, where, you know, have done the upload, but you would notice that you know, in Tik Tok or in YouTube everywhere, only, like 2% of all account holders are uploaders, or content creators and 98% are consumers because that's otherwise you know, none of these would work because people need to watch the content that is being created. So those 2% are never an accurate even though those two percent’s opinions are usually never the representation of general mass populations opinion and there is a big debate that usually goes within data scientists and statisticians that what percentage of the overall population should be an accurate representation of that, you know, total population. And there is there is always that debate, and I think that it's represented with a P value, which I'm not gonna go into too. Much detail, because I'm not very much expert on that. But it is a topic. Definitely an interesting topic to sort of explore further.

Amit Sarkar: 00:21:27

Well, I think, again, a good point because I watch a lot of tech videos on YouTube. And one of the biggest influences currently on YouTube, when it comes to technology is MKBHD. And he posts a lot of videos on smartphones and other smart gadgets. And one of the things that and one of the interesting things that he said in one of his videos was that I mean, there are content creators, of course, and as you rightly said, there are a few content creators and many consumers and that's how the whole system works. But then he said that because only few people are creating the content, you're getting really few opinions. You're not getting all the opinions out there. So, a lot of times when people say, why do you create a podcast? Why do you talk? When they asked me this question, I say it's just an opinion, right? And it's important to get the opinion out, and the more opinions you listen to the more rational decision you can come up to because sometimes what happens is you only hear one side of the argument or the you keep hearing only that side of the argument. Never hear the other side. So, like there is a moon conspiracies of People Keep Talking about the moon conspiracy. Where did we actually land on the moon? So, we say if someone listens to only moon conspiracy theories, then they reinforced that thinking and the same thing with visualization. If you keep visualizing the same kind of data, and if you keep visualizing the same kind of information, it reinforces your beliefs, your thought process. And that again, is dangerous. So you want to have different kinds of data, different kinds of information, and then you come up with an informed decision, because you can get the same information from various sources, but you can't trust which sources the right one, so like I can get the same use of the turkey earthquake recently from BBC from Reuters, from New York Times from an Indian

Rinat Malik: 00:23:18

or from an influencer on Instagram

Amit Sarkar: 00:23:19

influencer on Instagram, but then some, some would be hyping it up quite a lot. Some would be downplaying it quite a lot. So, you don't know whether it's actually serious or it's not very serious. And you have to then take an informed decision based on what you see, and what you hear and what you read. There is a very famous quote, which I keep quoting, I keep quoting it to everyone believe nothing of what you hear and only half of what you see it's a very good quote. And it just says that nothing that you hear should be trusted because you haven't seen it. And whatever you see, don't trust it completely, because it could be a trick. So, you always have to be careful even and it applies to this data visualization subject because whatever we see, you have to be very careful of what you're actually seeing unit of measurements, it's the most crucial thing. If you say the temperature is 100 What does it mean? 100 degrees Celsius or 100-degree Fahrenheit.

Rinat Malik: 00:24:14

It's crutial difference within that one would vaporize. One would sort of not, you know, one? Yeah,

Amit Sarkar: 00:24:23

so Exactly. So, the numbers are meaningless without the unit. So, a lot of times people got numbers, but they don't quote the units. Like recently MKBHD showed a graph of Apple showing any improvement of its chip over the previous generation. But they didn't show that it was for last year's chip, they showed that oh, the a 15 or a 16 bionic chip, is I think a 15 bionic chip is better than the last generation last the previous generation when they actually started the manufacturing or whatever. But they didn't actually say that the improvement over the previous generation was miniscule. So, they were comparing it with like couple of generations back rather than the previous generation. So of course, the graph would show that it has increased quite a lot the performance improvements, but actually with respect to the previous generation, the performance improvement was not that high, but then they need to sell more phones and they need to sell more phones based on Okay, we have improved performance we have improved the camera we have improved the battery. And that's how you trick people. And this was again highlighted in MKBHD. And he showed the graph, and that's where you think like, Okay, you're visualizing something, there is data, but what are you actually seeing? And that interpretation is important.

Rinat Malik: 00:25:34

Absolutely, absolutely. And one of the tricks that many businesses use and this is how they can get away with this because if you don't say anything, and if you leave it for the consumer to sort of imply upon themselves, then they're not in the wrong, they didn't give you wrong data, but they intentionally misled you because they didn't give you the unit or they didn't tell you the whole information, but they left it empty. Now leaving it empty makes a big difference because as humans we are designed to sort of fill the gaps that are that are there to sort of tell us the whole story our brain just does it. And we see it in optical illusions. We see it in many, many examples. And this is what's happening in data visualization. They're not. They're not giving you wrong data, but they're leaving some things intentionally blank. As a result, you're assuming something which actually might not be the case at all, and they're kind of designing it that way, which is, you know, intent intentionally misleading. But as long as they're not saying anything false, then there Okay, so that's, that's what we need to be extra mindful of when we're visualizing data. And it could actually sway our decisions quite, you know, in in a quite serious manner. For example, the tachyon earthquake you mentioned, you know, that could actually sway someone's decision of whether or not to donate and how much to donate based on the seriousness that they feel there is, and that's, that's an important, you know, that's a life or death situation for someone who is actually suffering based on the donation you know, people are doing worldwide. So, these things are actually you know, lifesaving in many ways or life destroying in in in the other way. So, it's very important that we always look at the asterisks and small letter, sort of lines to sort of give us that extra bit of understanding so we can make informed decisions.

Amit Sarkar: 00:27:03

Absolutely. And I think one of the key things about humans is that we are visual creatures. I mean, we tend to listen, but we also are visual creatures because we, when we, through evolution, we evolved looking at animals and we started hunting. We started protecting ourselves based on the weather. We looked up the sky we could navigate. So, we are basically visual creatures. And that's why visualization is important. If you show me a string of numbers and you tell me to make a sense of it. I will not be if you show me a table of data, I will not be able to make sense of it. But if you show me a graph, I can quickly make sense of it even though I don't know the numbers. So, I think that's where the power of data visualization comes because you can make quick decisions based on the visualization power because then you see a pattern and based on that pattern you can then take decisions. So, for example, I used to work for a client who was the telecom giant in the UK, and they used to collect the data and part of that data. You call it data warehouse, data mining, data analytics or whatever. But part of collecting the data was how to create new plans, so that you based on the consumer trend, so like how much people are consuming data, how much people are talking. So then what kind of plans you can come up with. And that's the power of like, data. I think we've already covered one topic on data previously, and the visualization comes here. Like okay, if I want to create a new plan, how many people are actually buying it and what's the trend like, is it going up? Is it going down, etc. And if I charge a bit more for data, but charge a bit less for the minutes that they're talking there, will it pay off? Or will it not pay off? So, you extrapolate in a graph or something? And that's where the business decision part comes? Because it's one thing to look, collect the data visualize it but then the other the last step is to actually take an informed decision. So, either you act on it, or you try to predict something based on that. So that's again, quite important.

Rinat Malik: 00:29:53

Yes, yes, absolutely. So, with technological innovation, we know we have all these different ways of visualizing data. And one of the things that I also want to add to this different sort of levels of visualization is we are and how virtual reality is helping us visualize data in a more detailed way. So, you know, you can nowadays in your computer screen to the screen, you can see a 2d plot x and y graph of a data oriented even see a 3d sort of data in your computer screen, you can rotate it around with your mouse or keyboard, etc, etc. But it would be so much more realistic if you actually went into a virtual reality world and then saw a 3d graph or a 3d plotting of some data in a way that you could sort of interact with, you know, as you know, by moving your hands and sort of amended to see which result we know which data would give you which results etc, etc. So as humans, you know how we visualize things and the more realistic the more interactive it is, the more we can sort of absorb it better. And you know, data information absorption is quite crucial if we're making decisions and predicting future actions. etc. So, virtual reality is, I think, the next step or another very important way of visualizing data. It doesn't have to be just limited to virtual reality, but also augmented reality so we could plot something in real life. And plot you know, some more data on top of it to see you can you know, you could potentially you know, look at you know, go to Dubai and look at Burj Khalifa and then also have on an augmented reality. See the second tallest building in the world which unfortunately, I don't know which one it is the CN Tower in Canada and have them compare right next to each other with augmented reality. So there are many ways of visualizing data, some of them are quite interesting, and, you know, even to, like a regular person who's not a data scientist would understand, you know, complex set of data through visualized, you know, to better visualization and if it's in 3d virtual reality world, it would be even more fascinating and sort of interesting to consume.

Amit Sarkar: 00:32:30

By the way, I think one of the things I need to clarify is that when we talk about data, it's just not numbers. It could be images, it could be videos, it could be text. It could be anything. That that conveys certain information when you look at it as a whole. So, data is just not numbers. When we talked about word cloud, it was not numbers, it was words, and then we look at heat maps. We are looking at images. Then YouTube creates like a visualization of the top 10 or 20 performing videos. So, data need not be in the format that you are familiar with. Because that's one of the things that we have grown up to like, okay, in schools, we use graphs, and in Excel, we use that. So, we think that that's just one data. And that's the only kind of data that we talk about. But as Renard mentioned, there are different things and different aspects to it. So, one of the things to take away from that is that data is not just numbers, it's many other things. It could be music as well.

Rinat Malik: 00:33:36

Oh, yes, yes, very much so and all of these senses that we have are consuming information and we could just as easily consume meaningful unstructured data through it. And our brain is really, really well designed to sort of find patterns within data. Sometimes it can be sort of wrongly used. If we keep seeing the same one-sided stories, but, you know, on a regular situation, our brain is quite well built for understanding or sort of absorbing data and make sense of it. So yeah, definitely. It was it was actually quite fond talking about this topic. And hopefully our audience enjoyed it as well. There was a lot of information and a lot of things to be aware of. Hopefully the audience would be now more aware of, sort of, dissecting the data that is presented to us every day, everywhere. So yeah, hopefully, this was informative and enjoyable. Please do let us know about your feedback or any other topic you'd like us to cover. We look forward to sort of hearing from you our contact details are in every platform, whichever one you're listening to us or watching us. So yeah, thank you again for listening. And we hope to see you again next week.

Amit Sarkar: 00:35:04

Thanks a lot. guys for listening and yeah, see you next week. Bye.

Rinat Malik: 00:35:08

Bye.