Analytics Interview Questions You Want To Ask Your Future Employer
If you're looking to get a new job in data science, you may want to evaluate the "data driven-ness" of the culture when interviewing. You'll save yourself the potential heartache of watching poorly designed hypothesis tests go out the door after you've informed stakeholders of the pitfalls. We often see tons of articles that discuss what hiring managers are looking in candidates, and candidates are studying and preparing for these questions to put their best foot forward. Interviewing is also a two-way street, there's a ton of information you need to collect for yourself as well.
Let's make sure that candidates are asking the right questions about the state of a company's data, so that they can make a truly informed decision about where they want to work. No one likes surprises. It's not that any of these questions would be a deal breaker necessarily, just make sure you know what you're getting into before you accept an offer.
There are many other questions I'd be looking to ask as well to assess the company culture. As my career has progressed, cultural fit has very much been the deciding factor when considering roles. It is absolutely important to work somewhere that you feel you can be yourself. After all, you spend a hell of a lot of time there. This article is not focused on assessing company culture, but is specifically focused on the questions you might want to ask to assess where a particular company is with their data transformation. By 'data transformation' I'm not talking about taking the natural log of one of your variables. I'm talking about the journey a company goes through while striving to modernize their approach to leveraging their data. Companies often start on a journey to become more data driven, and that's great! But it's also possible that you don't want to take a new job and find that all of their data lives in Excel spreadsheets rather than in a database (this exists folks).
Questions:
Ask: About the relationship with stakeholders. Who has the decision rights on final test design or analysis? Do stakeholders often make decisions that are counter to analysis?
I ask about who has decision right on analytics initiatives. Are analytics and their stakeholders thought partners who collaborate effectively? Or can raising a ticket for analytics feel like asking for food from a short-order cook?Would you like t-tests with that? With work, this relationship dynamic can obviously be changed. There's also the case where a position wouldn't have as much communication with stakeholders as typical data analyst and data scientist roles, there is always room for variability.
It is certainly worth clarifying in the interview whether you can expect your thoughts on methodology to be valued on day 1, or if there is significant work to be done to achieve that. It's probably important you understand the dynamics of this relationship before taking a role.
I also want to note, it's perfectly acceptable for an organization to make strategic decisions. These would be decisions counter to any analysis performed and you decide to do them anyways. Though I do believe that if that is the case your analytics resources should be focused on higher value initiatives and not on items where the final decision has already been made.
I Ask: About the state of self service tools for stakeholders. Are dashboards currently in place? Is there a lot of ad-hoc one off analysis (or are you working on high-value big impact projects)?
Do you love building dashboards? I enjoy making a nice dashboard occasionally, but if the stakeholders don't already have self-service data access, there's a high likelihood that this is what you'd be doing, and it's nice to know this in advance. Although I actually do enjoy building dashboards, the concern I have (and have experienced first hand) is when stakeholders cannot access the level of data they need, these asks now become "ad hoc analysis". These questions can be fun, it feels good to help someone find an answer, but they're typically not higher value questions. The time you help Mary from Marketing look at the price distribution of the cheapest items in the product catalog is not going to earn a spot on your resume. All positions will have ad-hoc queries, but you might want to make sure that ad hoc requests are a piece of the gig and not the majority of the gig. Ensuring that your future stakeholders can already access the data they need to perform their job will reduce the likelihood of constant ad-hoc requests.
Ask: About data governance. Do teams across the organization all have the same understood definition of the same fields or metrics? Or will you get a different answer for certain metrics depending on which department you ask?
Ever complete an analysis only to spend the next week digging into why your number is different from Jimmy's? It's not fun. When there's no governance in place, trying to get your numbers to foot with someone else can also be common. Being a data steward for tables leveraged by analytics was probably not the most exciting part of my job, but I'd take that anytime if the alternative was lacking data governance. When there are too many questions around how things are calculated, it's easier for people to change their mind about how things are calculated (to support their interests). My wish for you is a job where the data is correct and metrics are well defined.
Ask: Are they able to understand (or have data easily accessible) that would allow you to get at the full journey of the customer from first touch points all the way through attrition??
If you don't understand the customer journey, it's possible you're performing analysis on a disjointed view rather than the whole picture. Once you have the full picture of the customer journey, your previous beliefs, stories, and analysis might not hold up. Luckily, performing this analysis would lead to a ton of juicy insights and could be a ton of fun. It's also possible, that the organization you're interviewing with doesn't have the data to make this analysis possible. For instance, I've worked with companies that did not have website click data available. In many scenarios, it's hard to build predictive models without this, so much of what we care about in e-commerce is often tied to this behavior. Asking about the customer journey is my way of starting a dialogue about what data might be missing or hard to access. There's often a number of data sources that need to be integrated to go all the way from acquisition (and the channel that they came in on) all the way to churn. There's sales data, website click data, purchase data, customer service data, lots of data. This is all about having the relevant information you need to make an informed decision.
summary:
You can obviously still join an organization that has less than stellar answers to these questions. No one (or company) is perfect! My hope is you'll at least have a real good idea of what your job might look like and the challenges you might face if you choose to work for that company. The questions here are often the same problems we read about in blogs currently. Many data companies are still working their way through these! Once you take a job, remember to keep asking questions! I've written another post about asking great questions as a data scientist, you can check it out here.
Would love to hear your thoughts on questions that help you assess the current state of data at an organization.
Asking Great Questions as a Data Scientist
Asking questions can sometimes seem scary. No one wants to appear "silly." But I assure you:
- You're not silly.
- It's way more scary if you're not asking questions.
Data Science is a constant collaboration with the business and a series of questions and answers that allow you to deliver the analysis/model/data product that the business has in their head.
Questions are required to fully understand what the business wants and not find yourself making assumptions about what others are thinking.
Asking the right questions, like those you identified here is what separate Data Scientists that know 'why' from folks that only know what (tools and technologies).
-Kayode Ayankoya
We're going to answer the following questions:
- Where do we ask questions?
- What are great questions?
I had posted on LinkedIn recently about asking great questions in data science and received a ton of thought provoking comments. I will add a couple of my favorite comments/quotes throughout this article.
Where do we ask questions?
Basically every piece of the pipeline can be expressed as a question:
And each of these questions could involve a plethora of follow up questions.
To touch the tip of the iceberg, Kate Strachnyi posted a great assortment of questions that we typically ask (or want to consider) when scoping an analysis:
Few questions to ask yourself:
How will the results be used? (make business decision, invest in product category, work with a vendor, identify risks, etc)
What questions will the audience have about our analysis? (ability to filter on key segments, look at data across time to identify trends, drill-down into details, etc)
How should the questions be prioritized to derive the most value?
Who should be able to access the information? think about confidentiality/ security concerns
Do I have the required permissions or credentials to access the data necessary for analysis?
What are the different data sources, which variables do I need, and how much data will I need to get from each one?
Do I need all the data for more granular analysis, or do I need a subset to ensure faster performance?
-Kate Strachnyi
Kate's questions spanned both:
- Questions you'd ask stakeholders/different departments
- Questions you'd ask internally on the data science/analytics team.
Any of the questions above could yield a variety of answers, so it is imperative that you're asking questions. Just because you have something in your mind that is an awesome idea for approaching the problem, does not mean that other people don't similarly have awesome ideas that need to be heard an discussed. At the end of the day, data science typically functions as a support function to other areas of the business. Meaning we can't just go rogue.
In addition to getting clarification and asking questions of stakeholders of the project, you'll also want to collaborate and ask questions of those on your data science team.
Even the most seasoned data scientist will still find themselves creating a methodology or solution that isn't in their area of expertise or is a unique use case of an algorithm that would benefit from the thoughts of other data subject matter experts. Often times the person listening to your proposed methodology will just give you the thumbs up, but when you've been staring at your computer for hours there is also a chance that you haven't considered one of the underlying assumptions of your model or you're introducing bias somewhere. Someone with fresh eyes can give a new perspective and save you from realizing your error AFTER you've presented your results.
Keeping your methodology a secret until you deliver the results will not do you any favors. If anything, sharing your thoughts upfront and asking for feedback will help to ensure a successful outcome.
What are great questions?
Great questions are the ones that get asked. However, there is an art and science to asking good questions and also a learning process involved. Especially when you're starting at a new job, ask everything. Even if it's something that you believe you should already know, it's better to ask and course-correct, than to not ask. You could potentially lose hours working on an analysis and then have your boss tell you that you misunderstood the request.
It is helpful to also pose questions in a way that requires more than a "yes/no" response, so you can open up a dialogue and receive more context and information.
How we formulate the questions is also very important. I've often found that people feel judged by my questions. I have to reassure them that all I want is to understand how they work and what are their needs and that my intention is not to judge them or criticize them.
-Karlo Jimenez
I've experienced what Karlo mentioned myself. Being direct can sometimes come off as judgement. We definitely need to put on our "business acumen" hats on to the best of our ability to come across as someone who is genuinely trying to understand and deliver to their needs. I've found that if I can pose the question as "looking for their valuable feedback", it's a win-win for everyone involved.
As you build relationships with your team and stakeholders, this scenario is much less likely to occur. Once everyone realizes your personality and you've built a rapport, people will expect your line of questioning.
Follow up questions, in its various forms, are absolutely critical. Probing gives you an opportunity to paraphrase the ask and gain consensus before moving forward.
-Toby Baker
Follow-up questions feel good. When a question prompts another question you feel like you're really getting somewhere. Peeling back another layer of the onion if you will. You're collaborating, you're listening, you're in the zone.
In Summary
The main takeaway here is that there are a TON of questions you need to ask to effectively produce something that the business wants. Once you start asking questions, it'll become second nature and you'll immediately see the value and find yourself asking even more questions as you gain more experience.
Questioning has been instrumental to my career. An additional benefit is that I've found my 'voice' over the years. I feel heard in meetings and my opinion is valued. A lot of this growth has come from getting comfortable asking questions and I've also learned a ton about a given business/industry through asking these questions.
I've learned a lot about diversity of viewpoints and that people express information in different ways. This falls under the "business acumen" piece of data science that we're not often taught in school. But I hope you can go forward and fearlessly ask a whole bunch of questions.
Also published on KDNuggets: link