Facebook and data..

This is a post on Facebook data.. my Facebook friend list data..

Given I tell FB a lot, it was quite interesting to see how FB doesn't make it easy for me to even track what is going on with my Friend list and update me on what is changing. 

A few hours ago, I was trying to see how many of my friends post politial stuff on my feed. I could count 8 people off the top of my head and by skimming my feed over last week. Given it is holiday season, it is easy to spot the political posts.. My count was accurate. My entire feed is pretty much overwhelmed by these 8 people. Maybe the others dont post much? Maybe the most vocal ones are the ones that keep forwarding stuff to everyone on their timeline? Maybe the things they are forwarding are paid ads masquerading as meme's or just regular forwards? Maybe the people are not who they claim to be on FB and the account is hijacked? a lot of thoughts led me to try and find out again how after reducing my friend list to 150 two years ago, it is back at 318 now.. what has changed in the friend mix etc. etc.

Any analysis starts with the data set. Trying to export just the list of friends and the number of friends they have (or mutual friends) is not trivial. I did spend a lot of time copy pasting stuff from HTML to csv to JMP and had to do some clean up just to get the data in three Columns.. (Friends name, their friends total, mutual friends). Now why FB lists only Mutual friends in some cases I do not know. At first glance it looks like some folks do NOT want to even disclose how many friends they have!! I don't know what the point is of being on a social network if you are that worried. Let's move on..

Then I found that some of my friends have deactivated their accounts. Their pictures are blank and there was nothing listed. When I click on that friends page it says "this account is deactivated. you have the option to unfriend this person". WTH? This was for multiple people. So the person just left FB without a note of any kind to say they are deactivating it? 

72 of my friends show only a mutual friend count. So it makes it difficult to do a fair distribution.. that was a painful realization that there was no way to get the friend count. You have to go to that friends page and see if it will show up. Even my wife's page will only show we have 152 mutual friends. It wont tell me how many friends she has in total. Not that this count matters but if you are looking at data and you try to see patterns it makes sense to have apples to apples. 

Then I put in a classification by how they are my friends. 

F- Family (anyone related by blood or marriage)

HS- high school (PS. Senior)

C- college (classmates, juniors, seniors) from IT-BHU

GS - Grad school (All the Drexel and RPI connections)

FF- Family friends (this one is tricky because some of these people we met in social settings, my kids classmates, but Majority are spouses of friends from HS, C, GS or my wifes classmates and their spouses)

BL- friends from my Blogger days who I have managed to stay in touch with till now!

Y- Yoga (there are teachers and there are students, but I just lumped it into one)

W- Friends I made at Cypress Semi at work

Here are the distributions given this classification:

This one is the distribution of the friends my friends have (unfortunately for the 72 people out of 318 where it listed only mutual friends, that data is not included in this). 

If you read the anthropology books Guns Germs and Steel or Sapiens, the authors make a case that tribes split up into rival tribes or multiple tribes after their populations cross 140-150. The social structure at that point is unmanageable and the connections become meaningless. At any one point everyone closely interacts with 15 people on a day to day basis. In the last 2 years I have observed this at work, at home, at gatherings and on my social media feeds. People simply become inactive.

Given that there are some celebrities in my FB friend list (read star Yoga teachers, music teachers, public figures) I took them out of the distribution to see what it looks like for the bottom two bars in the previous graph.. ie, people with under 1000 friends.. The people who are between 600 and 1000 are mostly inactive but keep accepting friend requests and looking at some of their pages, I am seriously wondering if there are account issues. Again, companies that pride themselves on AI should be able to see patterns and alert people that one of their friends is multiplying on their friend count in an abnormal fashion that does not make sense.

117/212 still have <300 friends.. and 78/212 are within 200.. Gave this 150 number a lot of thought for the "average person".. read "not someone who is trying to get a following like a Business person or Brand ambassador)  and maybe given todays ability to connect more easily, that number can be 300.. It should be able to plot the number of active interactions between me and my friends to verify this. I am sure FB does this or has the capability to do this, but it would be nice for me to do it with data at least as it relates to my account and friends.

In the graph above the top four points on the Family (F) are wife, MIL, SIL and co-sister (see graph below) why it will only give the number of mutual friends for these four people is beyond me.  You can see that a lot of my High school and college friends are connected to each other by looking at the mutual friend count and it is the case with the Yoga community. I really don't have an apples to apples on this one as technically all the mutual friend counts are not available.

The graph below is the mutual friend list distribution where FB wont list their total friends but only mutual. Each bar is the number of friends who have a mutual friend list between those windows.. eg. 1 friend (wife) with 150-160 mutual friends, 1 with 50-60 (MIL), 2 friends with 40-50 mutual connections (SIL, co-sister), etc. etc.

The bottom 32 people with 0-10 mutual friends are mostly not sharing their friend list?

So what did I learn from all these exercises of graphing and charting and cleaning up data sets?

1- It is not easy to get exportable data out of Facebook.

2- There is no notification if your friends deactivate accounts or become inactive for prolonged periods of time

3- There is no warning if your friends account is showing abnormal increases in friend counts

4- A lot of my friends on the list have way more than 150 friends just like me. 

5- Never got to the actual reason why I started doing this.. which was to see what % of my friends post political stuff. That count is still 8/318 but there is no easy way to classify the posts on an excel file and put a code next to it and pareto it!  Out of almost 200 most recent posts in my feed, 50 were political and they came from 8 people.. They all seem to have posts on their pets, yoga, family which always seems to be anywhere between 50-70% of their posts. If there was a select way to mute that, I would be a happier person. 

The real question is "Am I seeing all their posts?" or "is FB only showing me select posts from these people that are political?" or "are these people deliberately posting this only to me and a select group?"

No way to tell. 

Majority of the posts are travel realated or family photos and pets. For that, I will gladly keep reviewing accounts periodically, clean up stuff and go explain to people why there is no point on being "friends" on FB when I don't get to see what they look like after 5 years but keep getting memes and political forwards for years. 

I am just having data withdrawal symptoms.. there is a lull at work and Christmas/New Year is coming.. next I will go look at my Yoga class graphs and charts.. Thankfully, that dataset was generated by me and is very accurate!

Faith Faith Faith

Faith is an interesting thing.. it makes you do things that are ingrained in you from childhood. Everytime I lose something and I have to find it in a time sensitive way, subconsciously say a prayer to Ganesha and say "will light a camphor light(karpooram) for 10 paise at the local temple".. 

a. we don't get camphor lights in the US that easily

b. there is no 10 paise anymore

c. local temple can be anything but was originally intended for Vembadi Vinayagar, a little ganesha under a pipal tree that I have circled with prayers to remove obstacles real and imaginary..

But it gets done.. and we are not even talking about if we actually found the thing on time or not.

In 2014 my MIL was going for a bone marrow test and in pure panic mode I prayed to my family deity (Gunaseelam perumal) that if the test came negative for cancer, will go shave my head in the temple. That was more of an extreme case of the camphor lighting thing.. 

Have made at least 6 trips to India since that time but never got the opportunity to go visit Gunaseelam. Faith is interesting.. seeing your family deity is not up to you, it is up to your deity.. or so the saying goes! Guess on this short trip to see my dad, there was no agenda. 

Three weeks ago, was lying in a hospital bed myself, dreading some worse case scenarios, which fortunately turned out to be something a lot less threatening. While there, the thought of unfinished business with god somehow bubbled up to the top of the already buffering thought process.  What if I died and did not do the "mottai"? After all the bone marrow test for MIL had come out negative! 

The original plan was to just go alone with a driver and get things done and get back. Then my parents said the words "Can we come with you?".. In case you didn't get the reason Faith got mentioned three times in the title, "you cannot say no to someone when they ask to come to a temple with you".. My dad was bedridden two weeks ago and he barely started walking again. An adult in diapers who just started walking, coming on a day trip where we spend 11 hours in the car and 3 hours in a temple is no joke. 

Somehow, we told ourselves, if it was meant to be, we will make it. I did a risk mitigation plan that would make any project manager proud and we left at 3AM. It was still raininig in Chennai but we were off to a flying start.. We had one stop for 25 minutes and were back on the road. 

Given this is my n'th mottai at Gunaseelam, the whole thing was done in 7 minutes.. then came the best part. There was water in the river that flows to the temple! Had a great time taking a dip in the river and have the fishes nibble at me. We were on a deadline and I had to pull myself out of the water to go in and finish the rest of the prayer. 

Our driver and some total strangers helped me navigate a wheel chair through the temple. The priest knows our family and was extremely nice and made sure my dad got to see the deity up close. 

On our way out a puppy came up and lay down under our vehicle. This one knew how to put on a sad face like a pro..

It was time to drive back. My dad is like a kid who just got his tonsils out and knows that his parents won't say "no" to anything... we just got out of the temple and he mumbles "elaneer.. I want elaneer". He wanted coconut water. Was not even sure if this is coconut water season and we were watching the road for it and sure enough we found a vendor selling fresh coconut water. It is not a good idea for a person of his age to drink that instead of lunch. We stopped at an A2B for lunch and it was really delicious. Had a full course meal. As an added bonus this place had a wheelchair ramp!

Then came the bombshell from my dad "I want to use the restroom". That was not part of the plan. It was going to be diapers all the way. But a kid without his tonsils... is a kid without his tonsils. The restrooms here are not typically wheelchair accessible. They are simply not for anyone who is not fit and can manoeuvre around a potty, a door that swishes past the potty with millimeter tolerance and a bunch of taps and buckets on the floor that are trip hazards. In what can only be described as a houdini move, managed to get my kid to go potty and get him diapy changed. That coconut water was a bad idea. Our driver concurred. 

The rest of the drive back was calm and serene. There were cops every 100 feet. So I started counting cops. After 270, I stopped counting. That was only on one side of the road from Dindivanam to Melmaruvaththur. My mind boggled at the efficiency of this whole security apparatus with so many cops just standing on the roadside doing nothing.. then again, maybe this system works, who knows! We actually slowed down to ask one of the cops to find out what the big deal was. He proudly answered "The Chief minister of Tamil Nadu is making his way back to Chennai on the same route!"

We made a dash for it to avoid his motorcade and made it back in exactly 14 hours. For a few minutes after reaching could not find my legs.. had let my dad doze off on me. . . which reminded me of my kids dozing off on me on another recent trip.

It has been another interesting day. 

A picture that says it all..

and a short video of the days experience.. 

It is always a great feeling taking a dip in the river.. mottai or not. Was glad to see my parents smiling again, even if for one day! 

Now the only thought in my head is "the mosquitoes are going to get me good tonight with this exposed scalp.. they will suck my brains dry and I might actually wake up smarter!"

My head feels lighter already, both literally and figuratively...

Great grandma visits and binge eating..

On our 5 day India trip, we managed to visit all great grandparents, thanks to some efficient auto rickshaws..

my maternal grandma is 90+ and her mind is very active. She is unable ot walk and she no longer asks me "when will my time come". she has been asking me that since my grandpa passed away almost 10 years ago. She has come to realize that no one knows.. and is now reading books to pass her time. She blessed us with all her heart!

San's grandparents are also pushing 90 and they are doing okay given their age. Walkers, hearing aids, etc. A slow methodical life and an approach of taking it one day at a time. Some lessons for San and me there.. Grandpa told me that till a few years ago he was doing yoga regularly including Nauli but now he is not able to. He is still active given his age and is a role model for all of us.

The best part was that they were the couple honored during my fathers sadhabhishgam as part of the "dhambati pooja" and given I was doing the pooja in proxy for my dad who could not bend down, it was a great honor! Will write a separate post on the function and what we did as part of it.. 

Then there was the eating in three places..

We went to watch Toy Story movie at the mall in 3D.. it has been some time since I saw anything in 3D given the massive headaches I get for days after watching any 3D movie.. but the kids wanted to go together somewhere and an Air conditioned theater was a good idea! After the movie the kids decided based on reviews that the best place that would accomodate everyones menu requests was Haven Sampoorna right near Phoenix mall in Velachery. It was an interesting place. I got my butter naan and side dishes and the kids who wanted pasta, got their pasta.. food was good. service was good. ambience was funny.. 

 Here is PIchchai Sundar with Sundar PIchchai... (kid you not, this was the photo right behind my chair)

you have no idea how many jokes I have to put up with ever since this guy became Google CEO. We are both the same age, went to similar schools and the same IIT coaching class in Chennai, think he also studied metallurgy, went to work for semi conductor companies ... I stayed in Semi and am traveling back and forth to Asia.. he moved to software and management..and probably has a lot of things going for him that I simply don't have or want.. but the jokes, they keep on coming.. So it was only fitting that I ended up below that photo! 

The sceond place we ate was Amethyst.. again nice ambience, relatively quiet and excellent food.. we went for evening coffee/ tea with San's cousin and brother's family who also happened to be visiting and ended up eating a lot of stuff..

The last place we went to was in Matsya in Adayar.. on a previous trip we had gone to another Matsya branch.. this was closer to where we stayed. It was also a nice place (we didnt take pictures). It was raining in Chennai that evening and we were all not our best when we got there. The funny thing with Ola/ Auto etc. is that they drop you off at the location but on the opposite side of the street and there is no way to cross over for a good kilometer either way! So we walked around a flyover for 25 minutes to get to the place. The dinner was great though. They had an amazing variety of dosas and cricket matches playing on 4 different TV's.. so I got to watch cricket and eat dosas.. 

That was pretty much all the activities we did outside of the function. 

Still waiting for photos from the function itself...

Missing San's maternal grandma who passed away recently as I write this post. We would always make a visit to see her as well. She passed away three months ago. 

Great grandparents are precious.. the way things are going, not a lot of kids get to see their great grandparents!