E037 Becoming R&D Data Scientist as a PhD in Twitter Analytics: From Industry to Academia and Back
Jul 26th 2022
Dr Neha Gupta holds a PhD in Data Science (2014-2018) awarded by the University of Warwick, Coventry, UK. In her PhD research, she applied data mining, geospatial analysis, and statistical Big Data techniques to understand the sentiment behind the social media (Twitter) data. Her research work tries to uncover the behavioural traits of people as perceived through social media channels. Before pursuing her PhD program, she worked as an IT professional in a variety of software development and test management roles building bespoke software for multinational companies (Bharti Telecom group, LG Soft, India, TEOCO and Avon Cosmetics) across India and the UK.
Coming from an academic family and inspired by her father, who was a Professor at an Indian University, she wanted to explore the possibility of pursuing an academic career. Therefore, to follow her aspiration she took a career break to take up PhD studies in Big Data in the UK. She did a couple of Data Science consultancy roles at HSBC UK and UCAS alongside her PhD since she was always passionate to explore the application aspect of research.
In the process of doing her PhD, she became an applied researcher that not only expanded her software development experience with interdisciplinary data science skills but also opened a new dimension of possibilities that exists in academia. She was able to perceive the challenges as well as opportunities available in Universities with a mature outlook.
During her PhD studies, she found her passion in teaching due to which she obtained Teaching Qualification required for the higher education sector in the UK and got awarded with Associate Fellow of Higher Education Academy. Soon after her PhD, she worked as a Teaching Associate at Warwick Business School which she recognizes as a very rewarding experience. Therefore, she continues to explore similar teaching opportunities in higher education where she could teach and share her knowledge alongside her applied research work in the corporate sector. Currently, she works as an R&D Data Scientist in AGCO Corporation, Stoneleigh, the UK where she is applying her research experience and cutting-edge data science knowledge to evaluate business forecasting models to predict supply chain demand trends.
In this webinar, Neha talks to us about her transitions: from several years of work in the IT industry to academia and back. Why did she choose to do a PhD and how does she values that experience? What are the challenges she faced as a mature PhD student in academia and how does she blend her industrial experience and networking skills with research knowledge to manage the expectations of PhD program?
She shares with us how she realized that the research-intensive academic publication path was not for her and that she would rather be suitable for either a teaching-based role in academia or in an applied R&D-focused role in the corporate sector. She shares with us how her PhD journey has been a transforming and self-realizing experience for her and how connecting and sharing ideas with people, and networking has helped her to create positive experiences in her life and career journey.
Neha’s LinkedIn profile: https://www.linkedin.com/in/neha-gupta-ph-d-8993815/
The episode was recorded on January 8th 2021. This material represents the speaker’s personal views and not the opinions of their current or former employer(s).
Natalia 00:09 Hello, everyone. This is yet another episode of career talks by welcome solutions. And in these meetings, we talk with professionals who share their career stories with us and teach us some life hacks. And today I have great pleasure to introduce Dr. Neha Gupta. She holds a PhD in data science awarded by the University of Warwick Coventry, UK. Before pursuing her PhD program, She worked as an IT professional in a variety of software development and test management roles, building bespoke software for multinational companies (Bharti Telecom group, LG Soft, India, TEOCO, and Avon Cosmetics) across India and the UK.
In her PhD research, she applied data mining or spatial analysis and statistical big data techniques to understand the sentiment behind the Twitter data, research work tries to uncover the behavioral traits of people as perceived through social media channels. Today, she works as a R&D data scientist at Oracle Corporation Stoneleigh UK where she is applying her research experience and cutting edge data science knowledge to evaluate business forecasting models to predict supply chain demand trends. And could you please tell us your story from your perspective?
Dr.Neha 01:21 Thank you very much, Natalia, for having me on board and it is a pleasure. Yes, I’m Neha Gupta as Natalia just introduced me, and I completed my master’s in computer science back in India in 2003 from a Central University in India, which is a Gurukul Gandhi University. And after that, I was working in telecom companies in India like party group, and LG soft India, working and developing the scope of the telecom software solutions.
My husband has been moved to the UK and I joined him in his journey to the UK then again I continue into my career in the telecom domain with Teoco in the UK again in a test management role building bespoke software but coming from an academic background to my dad was a professor and I was quite influenced by his personality and his knowledge in the way he conducted himself in life and I always had this dream to do a doctorate at some point, particularly from abroad, maybe from a country like UK or US to fulfill this dream.
And kind of follow the lineage where I come from so the moment I had my permanent residency sorted in the UK, I started looking for PhD roles within the UK, it was very challenging to secure a PhD position or an admission to a doctorate school in the UK because I didn’t have that research experience and just like anybody I Googled, I read people’s blogs on Quora on findaphd.com or jobs.ac.uk to find out how to apply for PhD positions, look for it and then finally, if I can get enrollment into a PhD program at a good university, that was where I was coming from.
Fortunately, I had a friend, who used to work with me in Teoco and she was a doctorate herself, and she did her PhD from UCL University College London. She introduced me to Assistant Professor in UCL and then fixed up a meeting with her because at this point, I was all over the internet to find a position to get admission, because my research proposal was not up to the market was pretty much the way I used to apply for jobs in the industry. I was kind of following the same approach. But I think there was a flaw in my approach. when I met this professor, the first question he said: Well, why do you like to do a PhD?
You are already working in the industry and there’s no point and you are moving ahead in your career, you are already in a team-leading role, and you will be a manager in the future. I shared with him that this is completely a personal motivation I had because I had a great influence of my dad on me, and also the kind of skill set brings so I want you to take a career break to do a PhD and experiment if I can pursue an academic career then what the PhD is like, just to have that experience.
He gave me a few keywords as he told me you can apply on these job portals, like findaphd.com but usually the transition, when people finish their masters in the UK or US, they go for peace of mind that is a unique journey if you like in that way. He gave me a few keywords like the Center of doctorate training, which was completely new to me, I was like, what is that? He explained to me that the Central of doctoral training is a kind of research group where universities secure funding for 4 or 5 years and they prepare a cohort of scientists if you like. But there is a slight training element involved in it, especially for the people who are transitioning into research roles.
He advised me that could be one of the good options for you, and then again back to Google, I looked for CDT positions. And fortunately, there was big funding secured by the University of Warwick at that point. I applied for it. And I think my industry experience gave me an edge there. Because there was funding in place for 10 PhD students, I was the 11th. when I applied, I was in touch with the CDT administrator Central of doctoral training administrator who was looking at the applications and he said, Well, we like your profile, your experience, and your cover letter is good. You seem keen to do a PhD, but your research proposal, it’s nowhere close to what we give this esteem funding to people and I was like, What should I do?
He said, well, why don’t you come and meet our Head of the department and have a chat with him and see what we can do for you then I had a meeting fixed up with the Head of the Department of Computer Science and fortunately maybe the universe was with me. He was very open-minded. He had worked in the industry for a few years in the US and then he moved back to academia. He kind of understood my passion and my urge to do it. I was not able to translate it well on paper, the way most researchers do. He gave me some guidance that what are the things they expect to see in a research proposal and apply again. I kind of followed what his advice was and fortunately, I could secure funding.
I’m funded PhD position for the first year and they said, based on my posterior PhD report, they’re going to reevaluate again. And then from the second year onwards, there could be funding in place. Fortunately, my first-year report was up to the mark. And then I got secured a fully-funded scholarship or PhD funding to do from the University of Warwick and the group was the smart cities group, quite an intimate, interdisciplinary kind of group. They were computer scientists there. They were people from a management background, who had to do their masters in management, they were also people from that background.
It was a very diverse kind of group. I fit it quite well there because I had some applied skills and some computer science skills at the same time. And I got awarded my PhD in 2018. I submitted and then obviously, the degree got awarded in 2019. After that, I did teaching at Warwick business school, it was a short-term teaching contract at Warwick business school, teaching a Big Data Analytics course, and currently, through LinkedIn, I got hired. And they forwarded my CV to a company called AGCO Corporation because they were building up a new data science arm. It is currently the R&D Data Science role in which I’m involved, where in I’m trying to apply all the data science skills that I’ve learned in my PhD in this new supply chain data science role. That’s what my introduction is if that helps.
Natalia 06:18 Thank you so much for this introduction. It’s a great story. And I love the fact that you were not intimidated to step in and out of science. And I have a lot of questions, first of all, my first question is, given that you were already building a career in the industry, Haven’t you had the temptation to go for one of these industrial PhDs? because I know that many companies these days offer their own PhD programs where you can stay within the company, you also have industry experience. you have both industry experience and also academic experience at the same time, plus salaries are accordingly higher often. My question would be, why did you decide to do your PhD at university?
Dr.Neha 09:46 There are two answers to that. I did find out at that point, when I started exploring my PhD, I was working for Avon cosmetics at that time, I was working on a big marketing data set. It would have been ideal if they supported my aspiration in that way. But what I figured out for an industrial PhD, was the timeline for that PhD was going to be quite extended, because obviously, it has to go on alongside my nine to five corporate responsibilities. when I explored that option, I figured out that it is going to be a part-time PhD, even though it is an industrial kind of a PhD.
That was in my case, and it’s going to last up to six to seven years, that is a big commitment. Already three to four years of PhD study itself is a commitment, especially the age when I started PhD was 35 or 36 so I think I didn’t want to commit myself for 7 or 8 years, that was the first reason.
The second one, there was this longing in me that I wanted to fully explore academia in the UK setting, or in a Europe setting, because I did my Master’s in India, and I never had an opportunity to go abroad and experience a university life abroad. It was an opportunity for me to understand how academia operates in European countries, but the UK particularly because I am in the UK. And I think I don’t regret that decision to take a career break, it was a challenge for one year because when I was not funded for the first year, I had to pay my fees and also had to live on my savings supported by my husband.
But from the second year onwards, it was almost 1/3 of my salary. It was good enough to pay the bills and support myself and the fees were paid for. I think from the second year onwards, it was all right. But in the first year, it was a challenge and I kind of even thought it was a full-time PhD. I managed to overlap the starting 3 or 4 months of it with my job. I reduced my working hours to 3 days. And then two days, I used to go to university, even though it was a full-time PhD but at that time, it was a transition phase and it was more of a literature review phase. I could manage that for 3 or 4 months, it was precisely 7,8 months. I would say that I was without salary if you’d like. These were the two reasons, I wanted to have that full-blown experience.
Natalia 11:22 I understand that. That’s also often the problem that indeed, some companies don’t want to pay you extra for the PhD experience. You have to kind of choose I understand. Now my question will be since Twitter becomes the big thing in the academic community, and actually, the academic discussion often shifts to Twitter, and then actually studying Twitter trends seems to be a dream topic for a PhD. My question will be what was your motivation to choose this topic? Did it happen to you? Just because it was a lucky coincidence, or perhaps you were interested in the sociology and all this whole social media phenomenon around Twitter or why did you choose this particular topic?
Dr.Neha 14:39 My PhD was in big data analytics, Natalia so one of the sources of big data is social media. Instagram posts, because we are bombarded with ever generating data. We are the consumers ourselves of data and we are the generators of data at the same time. Data is the new fuel as they say. I think, primarily, I was in a big data group Smart Cities group, which was the main influencing factor. Because in my cohort, there were 3 or 4 people who were researching big data analytics, particularly social media data, it’s Twitter and Facebook.
And the second reason, I would say I was interested in the behavioral science aspect of Big Data is quite interdisciplinary, I didn’t want it to go very deep into the machine learning route or pure maths because I didn’t have those skill sets. And I knew I would struggle. And I enjoyed the psychology as you mentioned, and the behavioral science aspect of it, how humans behave. There was a lot of existing research, new research coming up, using social media sources to monitor behavioral trends of people, and in particularly, the context of smart cities. because it’s ever generated data automatically, and this opportunity with researchers or slash scientists that we are in a position to mine this data and understand human rights as compared to the survey data, for which it takes months and years to collect that data.
And this data is organically generated, the availability of data and the hype behind it, the fact is that I was in this group, all contributed to choosing this group. And I think of that, these were the first three reasons as I said, the availability of data and my interest towards a behavioral science aspect, how do you monitor people’s behavior in Smart Cities domain, these are the three things, there was this opportunity to collaborate also with the existing researchers in the group. It was like, sharing knowledge, and there was this possibility to collaborate with one of the industry partners in my cohort, who was working on property data set, and the novel idea just came overnight, just the penny dropped in that, How about if we can link the Twitter data with this property data set, which is where my novel contribution lies to understand what’s happening in various industry domains.
My PhD is about understanding the tweets, which are coming from industry, and these industries, the way they are tagged by looking at what property addresses they are at. can we look at geotagged tweets to see? what sort of sentiment we are getting from different industry types? When I say different industry types, I mean, the construction industry, real estate, industry, and the financial industry. All these pieces are all these properties, they are at the backend tagged to some sort of SIC Standard Industrial Classification.
And if we know what kind of industry type and if we know the latitude, and longitude of a tweet, then there is some sort of connection to establish that this particular tweet originated from this kind of property? And is there carnism to kind of monitor the behavior of people belonging to a certain industry type? Think about, taking a survey of such people going to different this kind of industry and collecting the sample, it will be a mammoth task. However, using social media, it’s possible. That’s the whole idea, kind of ever.
Natalia 19:04 I have a private question about that. Which is in certain areas of industry, and general societies, Twitter is a major player that plays a role in terms of how the whole field is developing. For example, might be the blockchain where one influential figure with 100,000 or a million subscribers or followers on Twitter, actually solely can boost the price of our project by 50%.
By just even giving, untrue information that’s like the rumor is good enough to disturb the whole industry. If you understand Twitter, then you understand society in some ways and it’s a fascinating topic, and I don’t know much about this. But two years ago, in March of 2019, I was a participant in some hackathon project in Warsaw, we were doing our own brain hackathon, the second edition back then, I was just involved as a participant and the project was about analyzing Twitter data.
And I was not a big contributor there. But I remember that Twitter is quite welcoming to research on itself. They make APIs available, and you can website and they encourage researchers to research their own platform. I’m very curious, what types of information can you get? How deep can your insights about society be? How much can you understand the world and society from Twitter? Can you maybe share some interesting insights like either yours or like from other researchers, something that we as people who don’t do this type of research might never heard of? And that’s interesting when you think about society in general?
Dr.Neha 21:13 I think there’s a whole spectrum of topics. We have literally scratched the surface at the moment with social media data, and it is so dynamic, it is changing ever, so changing all the time. I think as far as my research was concerned, as I mentioned, I was looking at tweets coming from the industry. To answer your question, what kind of insights you can get from Twitter data? The person who is tweeting has shared the text, you can use or you can extract the text, you can extract the handle, you can extract to various hashtags, what they are following whom they are following whether it was retweeted.
This is all the textual-related information. But what’s also interesting is the geotagged element of it, which is the latitude and longitude. If you have caught the location services on your mobile phone, where you have installed the Twitter app, and you have enabled it to then the place from where you are tweeting, that becomes visible as well. That’s very interesting. Imagine, if you go back to survey days, you can’t keep on going chasing people, how they are feeling at a particular time at a particular place in a city.
For example, if they are in London, next to Big Ben, but with the tweet, what is happening, if they are linked with the Twitter and they have this geotag element to latitude and longitude information along with what they are feeling at that point, it will be really interesting that’s why it’d be a really interesting study to analyze various moods, during the various weather type or political chaos, like Trump kind of chaos is happening in the US right now. People are tweeting all the time.
You can generate all sorts of interesting insights from it. It’s mainly the one which I was exposed to or I particularly worked was on the geotagged element, the latitude and longitude, and the text of the tweet through which using various NLP Natural language processing methods, you can allocate a sentiment to it, whether it is a positive tweet or a negative tweet or a neutral related to it, looking at the words of the tweet, and then there is a whole realm of, body existing who are doing Natural language processing, particularly on Twitter, topic modeling, looking at various word characteristics, how these word vectors match to each other.
I just analyzed sentiment I didn’t go further in-depth to it. I just said, looking at the positive and the negative words and looking at some sort of lexicon or dictionaries. You compare these words and you allocate a sentiment to a tweet, whether it’s a positive tweet or it’s a negative tweet or it’s a neutral tweet and then I link that geotagged information latitude and longitude with the industry from where it is originating by using property maps.
It was a geospatial as well as text analytics kind of a job involved in that too. But to answer your question, you can generate a lot of business Real trends currently like for example, there are papers on people studying, political chaos, chaotic kind of situation behavioral trends, on, financial ups and downs when the stock goes up and downs and how people are following various handles on Twitter, whether that is influencing the stock going up and down. And then the mood of the people in the travel domain.
There was an upcoming study if we can manage the travel patterns. Although that was a failure, I would say, we were not able to manage it during the time of the pandemic, but people are tweeting all the time from airports. For example how their experience which flights they are boarding. You can monitor travel trends on it. This is all sorts of a huge spectrum of the topic of anything related to behavioral studies, and people’s emotions as they shed on to it.
Natalia 26:01 I was going to also ask about the stock exchange, since, from what I know, Google trends are one of the predictors that many investment funds even use to make moves on the stock exchange. Do you think that Twitter sentiment can also be a viable source of information to make investment decisions?
Dr.Neha 26:22 It could be because I think it was with regards to the Tesla stocks, the Tesla owner is quite active on Twitter. It depends, on who the person that influential person is, Jack Ma also or Alibaba owner. He was also quite active on Twitter at some point. It depends on how much following they have and how many people retweeted? what is their social media presence?
But, Google trained for sure. It businesses like the school itself, have published a few papers when they have used Google Trends to search keywords on Google to predict the stock market move, and there was coverage on, BBC News about it. Google Trends is quite the one basic, basically behavioral trends of people as they search for these keywords, the investors on the people who are buying stocks on Google, and if that can be used to tell whether that’s gonna go down or go up.
Natalia 27:37 I like to ask you one question that I think everyone who is watching this episode is now asking themselves, which is, how to build a following on Twitter? what are the deepest, darkest secrets of Twitter that we should know to be more efficient users of Twitter?
Dr. Neha 27:54 To be honest, I don’t know that answer. I’m sorry because you won’t believe I don’t have a lot of people following on Twitter. And I don’t follow a lot of people apart from a few conference groups and a few government tags if you like. Because mostly my job was to mind other people’s data, rather than me, following a lot of people or retweeting, my interest was always to extract data from Twitter, by using these APIs. And also, Twitter has got a commercial function that is very varying, they make data available for researchers. If you want to purchase a data set because one of the shortcomings with Twitter is that you mind free tweets, you only get, I think it was one or 10%, I can’t remember exactly, of the freely available tweets in a day.
You don’t have access to a full 100% sample of any particular handle or hashtag that you’re looking forward to. You’re only getting 1/10, or I think 1% I clearly can’t remember sorry, I didn’t realize that. It was a couple of years ago, I mind last time. What happens to those samples, which you mind from Twitter, for example, if your mind 100 tweets, it’s only 10 of those tweets, which are geotagged. If you’re interested in just looking at hashtags or handles or text of the tweet, you’re fine.
But in my case, I was interested in the latitude and longitude information, which is the geotagging of the tweet. I was able to get only 10% of my whole sample which I mind which was Geotag, when I kind of joined them in geospatially with my properties data to answer my question, it became even narrower. I had to increase my datasets. I purchased a dataset for two weeks, all Geotech from Twitter, the commercial arm and there is an agreement which you have to sign because they kind of retain the user information because of security reasons.
The user information is not revealed. But what you will get is the text of the tweet. And then you will also get, the Geotag, the latitude & longitude, you can comment upon a region in German rather than looking at the user characteristics as you kind of zoom out a bit about a particular trend in a city, that was my approach because obviously, you have to sign this agreement, you can’t use user information. Facebook was in trouble because their user data was led to so I had to sign up that agreement. I purchased two weeks of the tweet, which was all geotagged, it was funded by my department and that gave me a really rich data set.
There are two ways you can do with that post using historic KPIs, they’ve got a full Twitter developer page, you just follow the guidelines there. And then you get in touch with, there’s an email id provided and they gave you a script to mine that you’d have tweeted. I finally ended up doing that, because I wanted a bigger sample said because my study was based in London, and it was looking at two weeks of data in 2016, and two weeks of data in 2012.
Natalia 31:42 Great. And maybe, let’s come back to talking about your career. Because this is the width of this channel. I would like to ask you, since you first did the transition from industry, to academia, and then back to the industry, I like to ask you which one of these transitions was harder for you both, logistically, but also, mentally,
Dr.Neha 32:07 I would say, coming back from industry to academia, when I did my PhD, that was a challenging one, that was challenging, and because you tell me to compare these two, coming back to academia, I was coming back not for the job, but to do a PhD. But understanding the whole language of academia was completely new to me, to be honest, in terms of research grants, how the REF things were Research Excellence Framework in the UK, why publication is so important and I think, the way academia, and industry work because in the industry if you are writing a code, or if you’re developing a software, which is good enough, that’s the job keeps the customer and clients happy you are done. But in research, you get to the bottom of the question, and you try to uncover as much as possible in that analysis, and you carry on doing it until the results are achieved.
And data is less. However, in the industry, you’ve got lots of data, but you don’t have time, you don’t have that much drive, or there’s no such requirement to go to the depth of things to prove it until you are in a Google R&D lab But I was in a typical corporate job where I was building bespoke software as long as the bugs are not there in the software, the customer is happy, the software works, that’s fine. You can literally use a lot of APIs, but what those APIs are doing internally, you don’t need to comment upon those equations.
However, in academia when you’re writing a research paper or you’re working on a piece of analysis, you have to explain each and everything how you have achieved this result, what this method was from? where do you get this method? what is the reason behind it? I think that was a big challenging transition. It was mentally as well as physically challenging because I was not used to that kind of work. And I think working in isolation as well, because in the industry you’re working with a team you have your day job, but it’s in the interest of the whole team to make the project successful.
However, in academia, as you are doing your own PhD or working towards a paper or a chapter in PhD, it’s your responsibility, at the end of the day you are responsible for everything right from writing to analysis, validating, and finally publishing. Your supervisor is checking your work. But you are literally on the driving seat everywhere. That was a big challenge. Because I came from a team-leading position I had a team I was working with, and a lot of tasks were shared, there was a documentation team working on it. However, here, I was just a one-man army doing everything on my own. That was really challenging. But if you ask me, between these two transitions, coming back from industry to academia was very challenging.
Natalia 35:56 I can feel what you’re saying. And I had an individual PhD project, and I also experienced that isolation throughout my PhD. And it’s very weird because, on one hand, you are amongst people, you have those contacts, you are in a group you interact during lunch breaks, you go out of your office every day a few times, and you go to conferences in like, seemingly your environment. But at the end of the day, you are the one responsible for your own project. And it feels so heavy on your mind. I agree. And this is one of the main reasons why early career researchers leave academia, the isolation, and feeling of lack of real work. It’s one of the main reasons if not the biggest one, if not the most prevalent one, I totally understand.
But I also have to say that now having a company I can say that it helped in the long run. If I did that experience, I think now I would feel helpless because that’s exactly what you have to do. Once you start your own business. You have to do everything like you have to design the product yourself, do the work, but then also be your own marketer, your own salesperson, your own advocate, your own tax advisor, your own web designer, like everything together.
Dr. Neha 37:27 I can’t agree more. Because I think initially as you ask the challenge was there. And then you go through that journey when you are making that journey. It looks challenging and daunting experience. But once you come out of it, with few skates because of what that challenge has done to you it has a strengthened your muscles in many ways. And these are all sorts of muscles, your cognitive muscles, your perseverance, your surviving skills. I think that’s why at one point, I remember in the third year of my PhD I felt that I’m running my own business, I’m my own boss in many ways, and it is such a valuable skill to have in terms of if you can deal with lots of things single-handedly then it’s amazing how many people on Earth can do it.
I think that’s eventually what comes out of it while we’ll see what is in that journey. It is daunting, it could be shattering many times. But once you are out of it and once you have experienced it a couple of times, you learn coping mechanisms, and then you kind of plan well you set up your own deadlines in a way you become your own boss and manage in many ways, which is an amazing skill to have. Not just for your career, but even for life. You can deal with many things in a better way I can’t agree more because I feel I became very much independent in my time management in my work because earlier when I used to work in industry, it used to be like 9 to 5 at the most in the evenings I’ll check my emails, reply to my emails.
But now if for some reason if I’m reading a scientific paper and if my vote gets delayed, I don’t get tensed that much because I know that I have got that scale that I set once I give a final push and I’ll get it done because I know that I can do it within me on a day. Earlier it used to be remembered, trying to get the work done then the best the team is around. If you get stuck, what will happen but now that fear is less if I get stuck, I know I have ways to kind of manage it. Time overconfident and that fear is less. And then if you can do a PhD, there is always a way, it’s just that it requires a way, and you can search on the internet. you become very independent. I think I can’t agree more than that.
Natalia 40:20 Let’s talk a little bit about what you’re doing today. And my first question would be, first of all, you’re back in the industry. You started from the industry then went through the academic experience, and you came back to the industry? Does this experience of working in the industry and being a data scientist now differs in any way from what you’ve experienced before? Did PhD, added something to your mindset and to your skills that make your job easier now? Or make your decisions easier now. Do you feel any difference doing your industry job today, compared to what it was before your PhD?
Dr.Neha 41:01 I am completely a different person. Now, I have worked in the industry before my PhD for 10 years, and I used to think, I had a good length and breadth of experience. But until I did the PhD, I didn’t realize I had so many personal gaps, which I need to address, and the whole PhD journey has addressed some of those gaps, I wouldn’t say I’m not a complete human being after, you’ve been doing PhD, but it has addressed a lot of gaps. Acknowledging that you can provide a solution to everything.
And that fear is less the way I deal within the company, I am more confident to say no earlier, when I would be saying no, for any task, it’ll be like, it’s the end of the world, what will happen, Am I going to upset my boss, and what kind of knock-on impact it will have. But now, I think I’m very confident because science has given me an ever-evolving journey, be it in industry or academia, especially in the analytical domain, you can provide solutions to most of the world’s problems or real problems if they say, but you can do whatever you can do best of your knowledge.
And I can say that with a lot of information now, which was that information was not there earlier. I have better time management skills. And I think I’m a researcher because 3 or 4 years of independently invoking in isolation have made me a researcher. For example, last week, I was stuck in a code, I usually code in art, and I was like, how is it going to work? I was trying to manage a loop. But I knew, they would be held up label somewhere, I wrote a question on Stack Overflow, people replied, there, I tried various things. And this experimentation & trying, these are all the skills which have come from PhD, earlier I would do that, but there will be always this kind of fear of what happens if it doesn’t happen right now, so many diverse skills I have got because I’ve done teaching at Warwick business school as well.
I know, if I fail in this job, then there is definitely something out there for me where I can find a good job on my site, maybe policy, in the education sector, even in school teaching, it has left me enabled in many ways. There’s a lot of self-affirmation and confidence and less fear and more going out there and experimenting, and the way you interact with your team as well, the way you bring forward your own ideas. Because your supervisor kind of challenges you a lot in many ways. And it’s less challenging in industry, I would say because I had a were one of my supervisors was very ambitious. You become more resilient in many ways. I think it was amazing, 3,4 years of experience and I’m happy. I took a career break to experience it fully. But the way I work in the industry now, it’s different. And confidence and self-affirmation.
Natalia 44:36 I also asked because I’m aware that some people treat a PhD from the very start as an Army as a period of time when you go to get your butt kicked to have that experience of hardship and they do it on purpose and they do it consciously, not actually planning to become tenure track researcher, but rather getting this school of life and then getting out of the PhD and out of the Graduate School stronger, and, more self indeed self-sufficient.
Dr.Neha 45:18 I think that is something which I’ll have to mention, I think one of the key skills, which I gained out of PhD very key skill as a critical analysis because earlier, I remember, I used to bring in ideas, in a group of people, when I’m having team meetings, and all, maybe some people are naturally born, with that critical analysis element, but I wasn’t one of those. PhD taught me to question everything, and why do I believe what you are saying, just look at the more, 360-degree view, because not necessarily, that is the end of the role just because somebody is senior in your team, or somebody has spent several years in an organization doesn’t kind of state that you can’t bring in new ideas or innovation. And as I’ve mentioned earlier, not having a fear to say it out loud. That element of critical analysis is just because it has existed for 10 years.
The way of processes has worked just because it had it been that way for 5 years doesn’t mean it has to be like that in the future years also because you have seen the innovation side of it because you have learned as part of the research journey, how innovations are made, how you do the literature review, you kind of challenge the existing methods, you undo the science and redo the science. I try to do the same things in the industry now, which I don’t think will ever occur to me if I didn’t do the PhD.
Because naturally, this change is there. And not just in career life, but life in the journal also, I keep telling to my son, Do you believe that author, which you already don’t believe so he is getting it as part of my journey also. It’s a keep questioning, the things that you see around and, which you don’t agree to, you can always raise your viewpoint, your idea, which earlier, I wouldn’t think about.
Natalia 45:22 Fantastic. And if I can ask you now, a little bit more about your current job. Since R&D, data scientists is one of the hot jobs today for PhD. And it’s often an entry point to an industry career. So could I ask you just a few questions about how your daily life looks like so? Could you briefly summarize how your typical working week looks like in this job?
Dr.Neha 48:03 My typical work week, it is a data science role. In industry, the I’m engaged with the echo Corporation, currently, and they recently opened their data science arm, they’ve got this lot of supply chain data, and they have got lots of forecasting methods in place, they have these proprietary software which look at existing demand, and they try to forecast the demand in a better way for these supply chain parts, my day to day job is just looking at that data set and evaluating these forecasting methods, and if we can change various parameters, in this forecasting methods to even plan but just looking at the state of the art methods, and also what are the new methods, which are proposed by academia and in research in industry, which are similar to ours, and we better predict future trends.
Our methods are most optimized, and they kind of being commercially viable in a way and do help us in doing better business. That is one strand of it. That is, particularly in the data science role, wherein I’m looking at existing data sets and writing up new forecasting methods. And the second one is obvious, it’s the BI-related business intelligence. There’s a lot of reporting, on the existing data within the organization, and using Tableau dashboards for better business reports that are more of a front-end or KPI reporting (key performance indicators). Those are the two, jobs which I’m currently doing day to day.
Another thing is also going through literature in journals and in conferences with what are the new methods coming up in the forecasting domain? What are the new papers being published in supply chain business, following people on LinkedIn, that if there are new ideas coming up, we can bring those ideas and incorporate them into our practice. I think that’s primarily what my role has been so far, it is a very small team, we are just two and three people because it’s just literally started last year when they hired me, and the plans are to grow further. Because obviously, given it a supply chain, they have got a lot of data to analyze it in a more data science way and integrate lots of datasets from diverse sources. That we are looking at a more holistic picture, across the business. That’s what the role involves.
Natalia 51:20 Great. And do you have some long-term plans? Do you have some plan for your own development within your current workplace? Or maybe some more general plan for yourself in the long perspective, or don’t have to think too much about the future?
Dr.Neha 51:40 I’ve got academic experience, as well, I taught at Warwick business school for a short-term teaching contract, and I quite enjoyed that teaching role. It is working in the industry in this R&D role, but my ideal place would be if I can work with the industry, as well as if I can get some opportunity to do some sort of training within the company, or even outside as a freelancer somewhere, maybe over some weekend and I’m able to share practitioner knowledge as well as my research, knowledge which I’ve gained in past and even I read a lot now, I think, because I like that collaboration, I like that, knowledge sharing aspect of it, It’s always good to give it back you are what you are learning and you learn equally at the same time when you are sharing. I think that would be my ideal place.
Whereas I’m able to link myself up with some sort of training organization or even with academia be allowed to do some guest lectures maybe every now and then because that will keep me hooked on the academic side, as well as, industry because I don’t know whether I mentioned because I like quite applied research. And this job allows me to do research but in a more applied way, I can straight away see what the result of my analysis is, in terms of the trends we’re generating, rather than waiting for 5 or 6 months publication window, and then waiting to hear back what a peer review is saying about your researcher.
I quite like applied research in this kind of role. But I missed the teaching side of things at university because I taught undergraduate students as part of my PhD and also soon after my PhD. That was quite rewarding, the teaching experience was very rewarding. I would like to kind of have a flavor of it now and then just to keep me going because I enjoyed that aspect. I think these are the two things I would like to carry on going forward.
Natalia 54:00 I’m sure it will be possible sooner or later, there are so many options. Good luck. I wish you the best on your way. And do you also have any like regrets? I mean, is there anything that when you look back at your career, so far, if you had a chance, you would do differently?
Dr.Neha 54:25 Well, I think for the data science, because I think I could have spent more time learning more about coding I mean, because in PhD as I said, the biggest challenge which I face is because you are a one-man army, as I mentioned earlier, you have to write also you have to research to code to do the analysis, visualization and then submitted in the journal, and then on top of that, you are trying to take some teaching experience also.
There was a lot of my read of things you are doing in your PhD, although the main emphasis is on the researcher, I wish I could have spent more time doing some more hands-on coding. Because of these other things, sometimes they were a couple of months where I haven’t touched the code at all, and because I’m writing the paper and revisiting it, and then for 2 months, you haven’t looked at our code.
I think I would say, that is something not to regret, but that is something which I could have done better. If you want to have a career in data science, the data scientist role going forward, and coding is very important, the more you practice, the more hands-on you are, the better it will be for you when you are in the industry. It’s being aware of either the tools, I learned but a lot of people in my research group, were also doing Python, I wish I would have spent another 6 months learning Python as well.
Currently, my team here we both are using Python. It’s good to have a couple of hats or a couple of tools available for you in data science. Because the more you have, the better it is no need to be an expert in all of them. But it’s good to spend at least a good 6 months, knowing them and knowing the basics of at least two languages. That is going to go a long way. that is my key takeaway. I mean, I tried to be more disciplined now, I put my time aside now, to start learning Python, as well as I’m doing odd. But I think I had a little bit of time during PhD, which I could have done it. But then I was exploring the teaching side of things, I was fascinated by the university side. I think it would have been nice if I kind of indulged myself for a few months on it.
Natalia 57:08 I feel for you for what you’re saying. And this is interesting because when you ask this question to many PhDs, most of them would say, well, I could have spent more time on networking, or I could have spent more time on building my social skills, but I think I agree with you that it’s also very good to know like, what is your craft, and like, have a skill that is your technical skill, that is actually something like a hard skill that you have, and you really have to put it, learning to perfection and make it your craft so that everyone knows that you’re the specialist, you’re the best at this one thing, and even with one skill, you can get very far if you know how to use it in many contexts.
And for me, personally, this is not coding. I learned the hard way already. I don’t have a natural skills in four languages, including the programming languages. I can always learn where I’m decent, but I will never be the best and for me, what works best is writing, I can write all kinds of texts. And at some point, I had to decide, well, this is my craft. And if you have a craft, you have to keep on doing it, it’s a muscle, and you have to keep on stretching this muscle all the time.
Now, I’m occupied with writing all kinds of things, from essays for my blog, through books, some marketing materials for the company, and all kinds of texts I can find to write. I write and I totally agree, this is something you said at some point, do the self-observation and just figure out, that this is my craft, this is what I’m best at. And just keep on doing it until it’s not your personal best, but it’s you are the best in your community. And then that’s what makes you a top professional in your field.
Dr.Neha 59:07 Exactly Natalia. I mean, you’re 100% right. It’s about knowing what the craft is. With data science, because I am in the data science realm, the core skill is coding in many ways. You have to be a storyteller, and generate insights, which PhD gives you collaboration? But at the end of the day when it comes to analyzing data, coding is what will help you. There are a lot of packages of labels. when people say programming and coding, it could be scary, but I think something people viewers who are watching it, I would like to say I have done C programming myself when I started as a software developer back in 2004.
But believe me, data science programming, it’s not that daunting, like C programming technically could be because there are a lot of API’s available. And it’s a very high level of programming if you like, especially with R and Python, it’s just a matter of spending time on it, and you do get hang of it. As a data scientist, one of the core skills is knowing either of these languages, other languages are coming as well, somewhere on LinkedIn, you can find there are other flavors of Python and are coming but R and Python are pretty much in these days for data science, Tableau Yes, for visualization, you can say.
As a data scientist at the end of the day, even if you have to do storytelling, even if you have to do some collaboration, even if you have proposed a new idea, to produce a prototype, the very thing which is going to help you to produce a prototype is coding That’s very much center of it. And the more time you spend on it, the more comfortable you will be in applying it. I think, and as you said, it’s like a muscle, the more you do it, the less fear you will have about it, the more packages, you would know, you have a GitHub kind of repository, wherein all your code is sitting at one place helps you in finding the job, be very active on LinkedIn, have your LinkedIn profile up to date, because my first job which I got hired straight after PhD was through LinkedIn, I didn’t even apply for it.
I had links to my projects on LinkedIn, and something which I found just one ending note that earlier before we close this, I think CVs are very important. Because as part of PhD these were the tools that I learned in PhD I didn’t know about it before earlier. My CV would be in a Word document. But now as part of the PhD, I don’t know whether we use latex here. latex got changed, there’s an online free version of latex. What is it called? They have got lots of CV formats, their CV templates are there. And when I update so this was again, that independent PhD scale. I picked up a lot of CV templates from overleaf and I saw some of them were industry types, some of them were academic style and I had 2 CVs one for academia for my teaching job, which I applied and one was for the industry.
And every recruiter who shortlisted my CV, I got lots of interview calls, and everyone was like, your CV is amazing. I think that’s what PhD brings in, you learn what to highlight and what not to highlight. And these templates, which are there on Overleaf, you can emphasize the skill, the people are looking for and also translate your research skill into an industry more in you being independent, being innovative, these are all very much wanted skills in the industry.
Self-driving is just speaking those skills in a language that they would understand and selling yourself. That helped me a lot. With that Overleaf CV template, I can’t remember the name of the person who designed the template but thanks to him, give credit to him for his template. But that helped to kind of shape my experience into page CV format. And that’s your first impression, literally. And then, the rest is up to you how you perform in the interview or technical test?
Natalia 1:04:03 Right. This topic is also a bit controversial. You can hear a few different versions in the industry about what is the perfect format for a resume because also, big companies like that are so huge that they get hundreds of applicants per every opening. They often also pre-select resumes that are read by a human by running some software that is just, filtering out information. And sometimes these PDFs rendered from later are not very handy to the software. this can also prevent you from being hired, unfortunately. If you can submit a nice-looking CV that also has Like editable form, that’s perfect. In my case, when I was applying for jobs, if I had a choice, and I could choose to send two files, I was always choosing the nicely looking version that is like the Overleaf version, for instance.
But I was also submitting simple texts the same as the text version of the same resume just because I was never sure if they are using the software or not. I wanted also to make it convenient for them. It’s just a little remark from me that if you are applying for an international corporation, then often this is the case that they use the software. It would be good to also append your application with this simplified, very simple text format of your resume. But anyway, that was just sort of topic that offers amazing templates, I was also using them. I totally agree with that.
Dr.Neha 1:06:05 My last one more thing is never to undermine your skills. Because what has happened another thing which I would like to share from my personal experience, I think I appeared for an interview at one place, and you won’t believe I went in, I hope you allow a little bit of time because it’s a really interesting story. I applied for an interview, the position was applied for a data science manager role. I applied for it.
And I thought, why am applying for a data science manager role because I thought I got good data science exposure. During the PhD, I did a bit of consultancy, alongside PhD, because I work for HSBC at the time, had some experience in UCS, and had some industry experience before my PhD also, I thought, why not apply for the data science manager role, I do understand, a lot of times you learn on the job as well, they took my interview that went for 3 hours.
I was supposed to give a presentation to them on what new I can bring and what ideas I can bring with the data set, which they shared. And you won’t believe I’ve worked on that presentation for almost 2 or 3 days to give that a good holistic view of 360 degrees, I applied most of my research skills, and the kind of data sets and how you can integrate various data sets geospatial data, social media data, with this company, internal data, so that you can have these because it was a consumer-facing industry. A lot of social media could be used in that. I did that and the presentation went on for good 3-4 hours with a lunch break in between, and the director who was hiring me, he was like wow, these are some good ideas, because in the industry when they are working, I think that was an also it was 100 years old company. I think they didn’t have a lot of innovation in place.
A lot of the data sets, which I was talking about, it was completely new to them. I bring a lot of things. Unfortunately, that position got filled by an internal employee. The HR communicates to me that this position is being filtered by an internal employee but the director is keen to have you. we would like to call you again for another interview. And they created a new role called senior data scientist for me. That was high and this is what a PhD is giving you. You are being taken by the industry in a real positive way. And they have created a new position for you because they think you’re the right candidate. But what happened on the flip side, though, when I appeared for the second role, the guy who was interviewing me, was this newly posted data science manager.
Now, he was this traditional guy who has been working in the industry for the last 20 years, maybe knows a lot about his industry, but had a very limited experience with the new data sets, or new methodologies or innovation, which is happening in the data science domain. He was asking me one sort of question, and one of the questions he asked was, how long would you like to stay in the role? And I said, in a senior data scientist, or maybe a couple of years, and then I’d like to move on. By moving on, I meant that you don’t be promoted. I think that would be the obvious answer. who would like to be in a role for 2 years then you want to look for some career progression and Then he took a note of it. And then I get feedback from HR saying that the person has communicated that
she doesn’t want to stay with the company for more than 2 years. That’s why we are not selecting her.
And I was like, this is completely in contrast to the opinion of somebody who is of wisdom, like the director guy who created a position within the same organization, and somebody who was newly promoted to this position, but I believe my hunch says me that he got intimidated by the fact that somebody new is coming in their team who can challenge their own situation, because, at the end of the day, he will be my reporting manager. People are protective about their own jobs as well, which is fine, which is a human angle. I would like to share that experience, because, at that point, you think, Well, you did a PhD to kind of upskill yourself, get a more length and breadth of experience, but sometimes, if that’s not the right job for you, or if that’s not the right environment for you, where you can grow, then let it go.
I mean, I was upset for a while. But now I am in a team, my team is amazing, the director, whom I report to, he’s a very nice leader, he’s very forward-looking. You do get all sorts of people in the industry, who can be a little bit protective about their jobs, and they might find it challenging if somebody really, with full aspiration is coming in with lots of new ideas. But at the same time, they are people who want to grow their organization. Never take it on yourself, you have not done anything wrong to do a PhD.
And it’s a good skill. You are bringing invaluable skills, but you get all sorts of people. I wanted to share this episode because it was the whole interview process. And the selection process with HR went on for 2 months. There was a lot of communication, them creating a position for me, they creating, and then in the back in, there were other interviews happening. I was doing the teaching at Warwick business school. but these are the times when you have self-doubt coming in, but I think in the end if you have full trust in yourself, and your own knowledge. There is a place for you people who value that and you will be getting there at one time. I mean, you will land there. That was quite an interesting episode.
Natalia 1:12:59 I mean, I know if you apply to a large company, where a person who is interviewing you is not the decisional person because they’re not shareholders, they are not the owners, they are just HR Department employees, who are not even specialists in subject matter, just people who have like a list of requirements for who is the like our dream employee for this particular position, and what’s the profile and they try to fit to the profile and figure out if you match the expectations, and that said, they compare the two. And they try to find someone who fits the best to profile that they have. If you encounter people like this, your value or your ideas would not be rewarded the way they should be.
That’s also why for PhDs, especially if you have some very specific skills, or like bright personality, something, you’re thinking out of the box. In those cases, usually for PhDs, it’s more advisable to go to startups and get hired there. Because a person who will be interviewing you for a job is usually a founder himself or herself, or a shareholder, someone who is decisional and who has the company’s interest in their mind, primarily.
If they see the potential in you, even if you don’t feed the like the profile they created before the interview, they would likely change their mind because they see the opportunity. Whereas if you go to a corporation that what they assess is how well you fit the expectation and they don’t want to bet and they don’t want to gamble on it on the candidate, if you exceed the expectation, sometimes it goes against you, unfortunately. But the great story, so lastly, Do you have any other general advice for PhDs that you would like to share, some life hack, or some general cube that you will be willing to broadcast to our audience?
Dr.Neha 1:15:25 I think as I mentioned, Natalia, in my case, because I didn’t come with that research background, I left my masters a while ago, when I commenced my PhD journey, collaboration was a key to my success. At one point, there was this dearth of data and, ideas also not coming along. But just because I was part of the CDT group, I collaborated with my peers, and I came to know that his industry partners are showing the data set, and then I can purchase the data from Twitter. It’s just like, sometimes it is an isolated journey. But I had that skill from the industry because I used to work in a team environment. I employed that skill in my PhD, and that collaboration showed a path forward because you are making a path for yourself.
Even though it is an independent journey, it’s always good to talk to people network, within your research peer group because it is everybody’s interest. If you work on a paper, you are going to be the first author on the paper, if they can be a co-author, then it’s a win for both of you. It kind of moves your research forward. That was one piece of advice, I would say, be open to talk about it. Listen to YouTube’s video, there are times, when you are emotionally very low, but then you’re not alone. In this journey, everybody goes through the same feelings, especially if they’re doing PhD similar feelings if you like.
And the third one is like if you are aiming for data science, PhD coding is a key skill to have. You have to start with at least good hands-on coding rolling, even if you become a data scientist in R&D. For good 2,3 years, then only you could be in a leading position later on. But first, you have to try it hands-on yourself. If you are aiming for a technical data scientist role, then coding is a skill that required good to have exposure to one or two languages at a comfortable level and having a GitHub account. And that will help them in their journey. Good luck, keep smiling, it’s all worth it. When you are in the journey, even you don’t see light at the end of tunnel. You feel oh my god where you are stuck. But eventually, you will feel that it was worth it. It was worth it in my case.
Natalia 1:18:04 Fantastic. Thank you so much for joining us today and for sharing your story and your valuable insights on Twitter and careers possibility and your new job out as a data scientist. Thank you for everything. Thank you, guys, who successfully reached the end of this episode with us. Thank you so much for watching. If you’d like to get more of this type of content, then please subscribe to this channel and share your comments and questions with us. We’ll be happy to take them on board and answer every question. Thank you so much and have a nice day.
Are you planning to upgrade your career to the next level or change your career path? Are you pondering your options? Don’t be alone in the process – join us at our live online Ontology of Value® Career Mastery Program!
At this intensive online training, you will focus on discovering your identity as a professional, and learn effective career development strategies for landing great jobs.
We will help you choose the right career path, assist you in landing your new job, and teach you self-navigation strategies that will guarantee your success in professional development, and serve you for a lifetime!
Please find all the information about our incoming, game-changing program here:
Please cite as:
Bielczyk, N. (2021, January 17th). E037 Becoming R&D Data Scientist as a PhD in Twitter Analytics: From Industry to Academia and Back? Retrieved from https://ontologyofvalue.com/career-development-strategies-e037-becoming-rd-data-scientist-as-a-phd-in-twitter-analytics-from-industry-to-academia-and-back/
Would you like to learn more about how to thrive at work?