6/2/21 6/2/21

Analytics Interview Questions You Want To Ask Your Future Employer

If you're looking to get a new job in data science, you may want to evaluate the "data driven-ness" of the culture when interviewing. You'll save yourself the potential heartache of watching poorly designed hypothesis tests go out the door after you've informed stakeholders of the pitfalls. We often see tons of articles that discuss what hiring managers are looking in candidates, and candidates are studying and preparing for these questions to put their best foot forward. Interviewing is also a two-way street, there's a ton of information you need to collect for yourself as well.

via GIPHY

Let's make sure that candidates are asking the right questions about the state of a company's data, so that they can make a truly informed decision about where they want to work. No one likes surprises. It's not that any of these questions would be a deal breaker necessarily, just make sure you know what you're getting into before you accept an offer.

There are many other questions I'd be looking to ask as well to assess the company culture. As my career has progressed, cultural fit has very much been the deciding factor when considering roles. It is absolutely important to work somewhere that you feel you can be yourself. After all, you spend a hell of a lot of time there. This article is not focused on assessing company culture, but is specifically focused on the questions you might want to ask to assess where a particular company is with their data transformation. By 'data transformation' I'm not talking about taking the natural log of one of your variables. I'm talking about the journey a company goes through while striving to modernize their approach to leveraging their data. Companies often start on a journey to become more data driven, and that's great! But it's also possible that you don't want to take a new job and find that all of their data lives in Excel spreadsheets rather than in a database (this exists folks).

via GIPHY

Questions:

Ask: About the relationship with stakeholders. Who has the decision rights on final test design or analysis? Do stakeholders often make decisions that are counter to analysis?

I ask about who has decision right on analytics initiatives. Are analytics and their stakeholders thought partners who collaborate effectively? Or can raising a ticket for analytics feel like asking for food from a short-order cook?Would you like t-tests with that? With work, this relationship dynamic can obviously be changed. There's also the case where a position wouldn't have as much communication with stakeholders as typical data analyst and data scientist roles, there is always room for variability.

It is certainly worth clarifying in the interview whether you can expect your thoughts on methodology to be valued on day 1, or if there is significant work to be done to achieve that. It's probably important you understand the dynamics of this relationship before taking a role.

I also want to note, it's perfectly acceptable for an organization to make strategic decisions. These would be decisions counter to any analysis performed and you decide to do them anyways. Though I do believe that if that is the case your analytics resources should be focused on higher value initiatives and not on items where the final decision has already been made.

I Ask: About the state of self service tools for stakeholders. Are dashboards currently in place? Is there a lot of ad-hoc one off analysis (or are you working on high-value big impact projects)?

Do you love building dashboards? I enjoy making a nice dashboard occasionally, but if the stakeholders don't already have self-service data access, there's a high likelihood that this is what you'd be doing, and it's nice to know this in advance. Although I actually do enjoy building dashboards, the concern I have (and have experienced first hand) is when stakeholders cannot access the level of data they need, these asks now become "ad hoc analysis". These questions can be fun, it feels good to help someone find an answer, but they're typically not higher value questions. The time you help Mary from Marketing look at the price distribution of the cheapest items in the product catalog is not going to earn a spot on your resume. All positions will have ad-hoc queries, but you might want to make sure that ad hoc requests are a piece of the gig and not the majority of the gig. Ensuring that your future stakeholders can already access the data they need to perform their job will reduce the likelihood of constant ad-hoc requests.

Ask: About data governance. Do teams across the organization all have the same understood definition of the same fields or metrics? Or will you get a different answer for certain metrics depending on which department you ask?

Ever complete an analysis only to spend the next week digging into why your number is different from Jimmy's? It's not fun. When there's no governance in place, trying to get your numbers to foot with someone else can also be common. Being a data steward for tables leveraged by analytics was probably not the most exciting part of my job, but I'd take that anytime if the alternative was lacking data governance. When there are too many questions around how things are calculated, it's easier for people to change their mind about how things are calculated (to support their interests). My wish for you is a job where the data is correct and metrics are well defined.

Ask: Are they able to understand (or have data easily accessible) that would allow you to get at the full journey of the customer from first touch points all the way through attrition??

If you don't understand the customer journey, it's possible you're performing analysis on a disjointed view rather than the whole picture. Once you have the full picture of the customer journey, your previous beliefs, stories, and analysis might not hold up. Luckily, performing this analysis would lead to a ton of juicy insights and could be a ton of fun. It's also possible, that the organization you're interviewing with doesn't have the data to make this analysis possible. For instance, I've worked with companies that did not have website click data available. In many scenarios, it's hard to build predictive models without this, so much of what we care about in e-commerce is often tied to this behavior. Asking about the customer journey is my way of starting a dialogue about what data might be missing or hard to access. There's often a number of data sources that need to be integrated to go all the way from acquisition (and the channel that they came in on) all the way to churn. There's sales data, website click data, purchase data, customer service data, lots of data. This is all about having the relevant information you need to make an informed decision.

summary:

You can obviously still join an organization that has less than stellar answers to these questions. No one (or company) is perfect! My hope is you'll at least have a real good idea of what your job might look like and the challenges you might face if you choose to work for that company. The questions here are often the same problems we read about in blogs currently. Many data companies are still working their way through these! Once you take a job, remember to keep asking questions! I've written another post about asking great questions as a data scientist, you can check it out here.

Would love to hear your thoughts on questions that help you assess the current state of data at an organization.

7/2/18 7/2/18

How to Ace the In-Person Data Science Interview

I’ve written previously about my recent data science job hunt, but this article is solely devoted to the in-person interview. That full-day, try to razzle-dazzle em’, cross your fingers and hope you’re well prepared for what gets thrown at you. After attending a ton of these interviews, I’ve found that they tend to follow some pretty standard schedules.

But first, if your sending out job applications and aren't hearing back, you'll want to take a second look at your resume. I've written a couple articles on how to create a strong resume. One helpful article is

here.

You may meet with 3–7 different people, and throughout the span of meeting with these different people, you’ll probably cover:

Tell me about yourself
Behavioral interview questions
“White boarding” SQL
“White boarding” code (technical interview)
Talking about items on your resume
Simple analysis interview questions
Asking questions of your own

Tell me about yourselfI’ve mentioned this before when talking about phone screens. The way I approach this never changes. People just want to hear that you can speak to who you are and what you’re doing. Mine was some variation of:I am a Data Scientist with 8 years of experience using statistical methods and analysis to solve business problems across various industries. I’m skilled in SQL, model building in R, and I’m currently learning Python.

Behavioral Questions

Almost every company I spoke with asked interview questions that should be answered in the STAR format. The most prevalent STAR questions I’ve seen in Data Science interviews are:

Tell me about a time you explained technical results to a non-technical person
Tell me about a time you improved a process
Tell me about a time with a difficult stakeholder, and how was it resolved

The goal here is to concisely and clearly explain the Situation, Task, Action and Result. My response to the “technical results” questions would go something like this:Vistaprint is a company that sells marketing materials for small businesses online (always give context, the interviewer may not be familiar with the company). I had the opportunity to do a customer behavioral segmentation using k-means. This involved creating 54 variables, standardizing the data, plenty of analysis, etc. When it was time to share my results with stakeholders, I had really taken this information up a level and built out the story. Instead of talking about the methodology, I spoke to who the customer segments were and how their behaviors were different. I also stressed that this segmentation was actionable! We could identify these customers in our database, develop campaigns to target them, and I gave examples of specific campaigns we might try. This is an example of when I explained technical results to non-technical stakeholders. (always restate the question afterwards).For me, these questions required some preparation time. I gave some real thought to my best examples from my experience, and practiced saying the answer. This time paid-off. I was asked these same questions over and over throughout my interviewing.

White Boarding:

White Boarding SQL

This is when the interviewer has you stand at the whiteboard an answer some SQL questions. In most scenarios, they’ll tape a couple pieces of paper up on the whiteboard. I have a free video course on refreshing SQL for the data science interview

here.

White Boarding Code

As mentioned in my previous article. I was asked FizzBuzz two days in a row by two different companies. A possible way to write the solution (just took a screenshot of my computer) is below:

The coding problem will most likely involve some loops, logic statements and may have you define a function. The hiring manager just wants to be sure that when you say you can code, you at least have some basic programming knowledge.

Items on Your Resume

I’ve been asked about all the methods I mention on my resume at one point or another (regression, classification, time-series analysis, MVT testing, etc). I don’t mention my thesis from my Master’s Degree on my resume, but casually referenced it when asked if I had previously had experience with Bayesian methods.

The interviewer followed up with a question on the prior distributions used in my thesis.

I had finished my thesis 9 years ago, couldn’t remember the priors and told him I’d need to follow up.

I did follow up and send him the answer to his question, they did offer me a job, but it’s not a scenario you want to find yourself in. If you are going to reference something, be able to speak to it. Even if it means refreshing your memory by looking at wikipedia ahead of the interview. Things on your resume and projects you mention should be a home run.

Simple Analysis Questions

Some basic questions will be asked to make sure that you have an understanding of how numbers work. The question may require you to draw a graph or use some algebra to get at an answer, and it’ll show that you have some business context and can explain what is going on. Questions around changes in conversion, average sale price, why is revenue down in this scenario? What model would you choose in this scenario? Typically I’m asked two or three questions of this type.

I was asked a probability question at one interview. They asked what the expected value was of rolling a fair die. I was then asked if the die was weighted in a certain way, what would the expected value of that die be. I wasn’t allowed to use a calculator.

Questions I asked:

Tell me about the behaviors of a person that you would consider a high-performing/high-potential employee.

Honestly, I used the question above to try and get at whether you needed to work 60 hours a week and work on the weekends to be someone who stood out. I pretty frequently work on the weekends because I enjoy what I do, I wouldn’t enjoy it if it was expected.

What software are you using?

Really, I like to get this question out of the way during the phone screen. I’m not personally interested in working for a SAS shop, so I’d want to know that upfront. My favorite response to this question is “you can use whatever open source tools you’d like as long as it’s appropriate for the problem.”

Is there anything else I can tell you about my skills and qualifications to let you know that I am a good fit for this job?

This is your opportunity to let them tell you if there is anything that you haven’t covered yet, or that they might be concerned about. You don’t want to leave an interview with them feeling like they didn’t get EVERYTHING they needed to make a decision on whether or not to hire you.

When can I expect to hear from you?

I also ask about the reporting structure, and I certainly ask about what type of projects I’d be working on soon after starting (if that is not already clear).

Summary

I wish you so much success in your data science interviews. Hopefully you meet a lot of great people, and have a positive experience. After each interview, remember to send your thank you notes! If you do not receive an offer, or do not accept an offer from a given company, still go on LinkedIn and send them connection requests. You never know when timing might be better in the future and your paths might cross.

To read about my job hunt from the first application until I accepted an offer,

click here