Improving Higher Education Through Data with David Niemi

Listen to the Episode

Subscribe to the Podcast

Episode Summary

David Niemi loves higher education. So much so that he’s spent his entire career involved in it. From an early age, David recognized that there were better ways to help students and learners achieve their goals, and he’s been on a mission to make that experience better ever since. Throughout his career, he’s been a teacher, student, EdTech leader, professor, and analyst. David perfectly straddles the line between technology, data science and education, which makes him well suited for leading Kaplan’s Learning Analytics division, as the VP of Measurement and Evaluation.

EdTech has come a long way in the last 20 years. But even today, David believes there’s lots of opportunity to do it better. He starts with a basic question: “If we actually built ed-tech that taught people something, how would we know if they’re actually learning anything?”

This is the foundation for David’s role at Kaplan. He’s looking past “completion rates” and “GPA” and looking at measuring the real skills that are transferred to the students. He’s focused on the learner outcomes, like getting a job, increasing their salary, and improving their lives and communities.

So what are the key metrics or questions should Higher Ed be focusing on? David boils it down to three easy points: Are the students learning something? What is the level of student engagement during the course? What are the measures of student motivation throughout the course? These are different than the typical metrics because they are collected in near-real-time and provide teachers with tailored feedback on each student that ensures they’re getting the right level of instruction at the right time.

“A measure of learning should tell you what new skills, knowledge, ideas and concepts have you developed. Not how many courses you completed.”

— David Niemi

David also shared some interesting correlations between how to successfully educate learners and how to run successful analytics projects. In both cases, you need start with the end in mind… For education, it’s

1) what do you want to do in your career?

2) What skills do you need to get there?

3) Which classes or programs will provide those skills?

This is exactly how analytics projects should work!

1) What does the business need to solve?

2) What data do we need to inform those decisions?

3) What techniques do we use to tease the answers out of the data?

We also talked a bit about David’s new book, Learning Analytics in Education which is a set of research studies focused on pairing education data with data science techniques to drive better engagement for students, whether in online classes or in-person.

The book is one of the first to look at these new EdTech platforms that allow for ongoing measurements of student progress. They investigate how they can use these new data points to help educators increase their students’ success. These educators can now harness data to personalize the experiences for learners, while improving overall outcomes at scale.

If you’re at all interested in this brand new space, we strongly encourage you to pick up a copy!

And thanks to David for coming on the show!

More about David

Check out David’s book on Amazon: Learning Analytics in Education

Connect with David on LinkedIn: in/david-niemi-2630757

Follow Kaplan on Twitter: @KaplanNews

---

Who is Driving Your Data Culture Transformation

There are several critical roles that are critical to increasing the maturity of your analytics. But the glue that holds it all together is the person we refer to as the Data Champion. You won’t see a job description for a “Data Champion”, but all organizations that have a strong data culture will have at least one, and likely more.

Data Champions are people who spearhead data culture within an organization. Sometimes, they are an executive or senior leader, but oftentimes, they’re the boots-on-the-ground people who are simply passionate about promoting and improving data-informed decisions for their organization. They may be part of a business team, data team, or technology team. They may be extroverts or introverts.

What does a Data Champion look like?

Data Champions are natural disruptors, communicators, and networkers who can establish, drive, and support a clear, data-informed vision. Data Champions are made, not born. You will often spot them because they will be seeking to start an internal meetup around a data-related topic or starting a data visualization competition, or maybe they will be the person who is at another’s desk showing them how to approach a data problem. They aren’t necessarily the most technical person in the room. But they are most certainly the ones who are building communities, telling stories about the possibilities, and focused on embedding analytics into every corner of the organization.

These Data Champions will be present in an organization whether they have been sought out or not. Organizations with a strong Data Culture, though, will have more of these Data Champions, and their level of empowerment and satisfaction will be higher.

What does a Data Champion do?

Data Champions play a key role in helping translate between the business and their area of the organization to help drive data usage when making decisions. They engage with business and technology partners to ensure they are smoothly working together. Further, Data Champions will have relationships with other current and future Data Champions, including those not within the Chief Data Officer’s direct area.

Data Champions are more than just translators though. They create vision, alignment, and empowerment for the teams they support. They build energy and excitement for a data-informed approach. They are skilled at working with business leaders to build trust in the analytics solutions being built. They constantly communicate the benefits that data can provide and the results that the organization has gotten from analytics investments, and they communicate the vision for the future.

Champions are Critical but not sufficient

Getting the organization moving in the right direction is obviously important. However, doing so without executive buy-in will result in frustration, limited results, and a lack of funding. Executives have to be part of the equation.

Similarly, moving forward without a technology foundation (quality data, storage platforms, reporting tools), and skilled analysts to dive into that data, will also result in limited results and frustration. The data team and technology must be a critical part of the equation.

Finally, it’s important to note that the best champions are the ones who work themselves out of jobs. “Translating” between the business and analyst teams is critical in the early going. But think of the benefits of translating; it didn’t need to happen, and both teams simply spoke the same language. Reduced friction, reduced effort, and faster/clearer communication would result. The data champion only translates until they can get the teams talking in the same “language”.

Here’s a great video about how Data Translators are critical pieces but are a stepping stone to the whole organization being data literate.

Credit: McKinsey & Company

---

---

---

The Key Roles of a Data-Informed Organization

Most people would all agree that data is a key go-forward strategy for their organizations.

As we discussed in a recent article however, there are some significant challenges that come with executing on that strategy. How do we overcome those? You need to embed analytics and data science directly into your organization’s culture.

There are three interlocking roles, each with some level of responsibility for making analytics work. The fourth and most critical role, the “Champion” sits at the center of these roles, driving alignment between everyone and driving successful change managment.

View fullsize

Over the next couple weeks, we’ll break down these roles in much more detail, but here’s a high-level overview:

The Executive Team

The CEO, CFO, CMO, CHRO, an the rest of the C-suite. When push comes to shove they need to support data initiatives, support the financial investment in the, weave data into the strategies of the organization, and ultimately hold the organization accountable to data-informed decisions and actions.

The Business Team

The many core functional areas of your organization. From human resources to sales to product to finance, the business team is critical to driving successful analytics. They must be on board and empowered to use data. Without this team informed, engaged and comfortable with data, then your amazing analytics outputs will fall on deaf ears, and the potential business value will be lost.

The Data Team

The extremely adept technical team who will be moving, storing, touching, analyzing, manipulating, and communicating your data. There are many roles within this broad category, but could include people like BI Developers, ETL Developers, Business Analysts, Data Scientists, and Report Creators. The key to their success is to turn them into key business partners, rather than basic order takers.

The Data Champion

The highly driven person or team at the center of it all. They are the evangelists that shout from the rooftops the importance of data for your organization. They “translate” how data can help the business, communicate it to leadership, and ensure the data team executes on the efforts. Data Champions are natural disruptors, communicators and networkers who can establish a clear data-informed vision. They create excitement and energy around data, and know how to influence the other three groups on how to execute.

These stakeholders together provide the pillars of support for an organization’s data culture. If one or more pillar is out of alignment, then the whole data culture is weakened. One pillar is not more or less important than any other. They each play a role in driving the data maturity of the organization and in-turn, the value that can be captured by analytics.

So what about your organization? Can you identify the people who fall into each of these groups? Are each of them in alignment with each other? What is the missing link that is holding your organization back from leveraging data effectively?

---

Need help?

---

Stories from the Data Trenches with Liz Weber and Tessa Enns

Listen to the Episode

Subscribe to the Podcast

Episode Summary

Back in December, Dave and I had the privilege of doing a speaking event at the MinneFRAMA (Finance, Retail, Marketing Analytics) event. Naturally, we decided to try something different and tape a live episode with two of my favorite analytics professionals in the Twin Cities, Tessa Enns and Liz Weber.

It was truly a once-in-a-lifetime experience: Sipping coffee, and talking analytics with these two amazing women. The venue didn’t hurt either! We were in a huge room at the Minnesota Science Museum, with our backs against a wall of windows overlooking the Mississippi river. We learned a lot about how to make sure your analytics projects are truly successful.

Tessa Enns

“I’m not working to fuel my technical skills, my technical skills are working to fuel the business challenges”

— Tessa Enns

Tessa talked to us about “accidentally” coming into an analyst role at Cargill, being given a huge transportation dataset and being tasked with finding something in it. Tessa is the kind of amazing person who looked at this as an opportunity, and went right to work, learning the data, learning the business, and learning technical skills along the way.

What is so amazing about her journey is that she was able to build a strong relationship with the business, who now trust the data, find opportunities to improve, and know how to turn the numbers into action that drives real monetary value.

Tessa preaches an approach where analysts need to “lead with the needs, not with the data”. She says this helps the analyst understand the real problem and help solve it. She also recommends putting every insight into dollar terms that your business will understand. “I put the cost savings or cost impacts right at the top of every dashboard”.

Liz Weber

“Take some time to understand the business processes that create your data. You will be able to understand your business teams and tell better stories that drive change”

— Liz Weber

Liz talked to us about a highly complex pricing challenge that her VP faced a number of years ago. The team was going through a major transition, and had invested a huge amount of money in their business. The VP wanted a dashboard to start monitoring whether this transformation was going successfully.

Liz started with the end in mind. Learn about what the leader wants, where they’re trying to go, and what the key measures of success really look like. She then sat down with the IT team, and the business teams underneath this VP. Making sure that everyone had a voice in the project was mission-critical to make sure that she 1) had the right resources, 2) had everyone moving in the same direction, 3) made the solution better than just her own ideas. It also helped with adoption, and making sure that everyone actually USED the end product.

What she learned from this project was that your leader/sponsor matters. If you don’t have the right sponsorship, it doesn’t matter how smart you are, or what kinds of technical knowledge you possess. You need to make sure you’re aligned to leaders who are committed to doing something with the outputs you produce.

Resources & Links

• Tessa on LinkedIn: in/tessa-enns

• Tessa on Twitter: @TessaEnns

• Liz on LinkedIn: in/lizweber2

• Twin Cities Tableau User Group

• Twin Cities Alteryx User Group

• Minneanalytics

• Cargill

• UNFI Supervalu

---

Five reasons why Data Culture is just as important as Data Science

Data Science is an amazing tool in the organization’s toolbox. It can provide immeasurable value when done well. Those in data circles have spent the last 10 years hearing about those success stories, reading example after example of the great things it can create. To be sure, there is still untapped value in that “oil well”.

And yet, while executives have been steadily investing more resources into tapping this seemingly endless well of value, we’re starting to see the cracks. Gartner estimates that by 2022, 20 percent of analytic insights will deliver business outcomes. Wait, what? If that’s the future, what does the current state look like?

Anecdotally, you see it too. The business is frustrated that they’re not seeing the returns. The CFO is starting to scrutinize those budget lines closer. The CEO is getting more impatient for results.

Is this a failure of the data scientists? Maybe we couldn’t afford the “good” ones. Perhaps they didn’t do enough data visualization or data storytelling?

Perhaps it’s a failure of the executives? They didn’t invest as much as they needed to get the return. Or, perhaps they weren’t fully bought into this new approach.

Perhaps it’s a failure of the business teams? They didn’t value what the data scientist could bring to the table. Or, perhaps they didn’t listen to the recommendations and insights being generated.

Or worse yet, data science just doesn’t provide the value we thought it did. Yes, people are actually asking this question.

Data is not a siloed activity

Separating the data teams from the business teams is a surefire way to never get value from your data

I once worked with a brilliant analyst named Dan. He was a true data scientist before that term was popularized. A truly skilled mathematician, gifted with just about any statistical tool you could give him, he could quickly diagnose and solve the stickiest data challenges we had. I learned a lot from Dan early in my career.

But Dan wanted nothing to do with the business team he supported. Meetings with the business partners was a nightmare for him. He was clearly intellectually superior to those regular old business people. He knew what they needed and he was going to provide them with that. Besides, he was on the analytics team and didn’t report to them, so it didn’t really matter what they wanted.

Perhaps not quite as extreme as Dan, but I often come across this “Analytics vs. Business” mentality in analytics teams that I work with. And I think the root of the problem is that we’ve structured our teams in such a way that we’ve isolated them from each other.

How many of your organizations set out to build more analytic capabilities by hiring a “head of” analytics, then building out a team, then going to work building (or re-building) the data infrastructure? I’ve personally worked with a dozen Fortune 500’s that fit this bill. It’s not a bad place for a startup analytics group. Silo your efforts so you can focus on laying the groundwork. Afterall, you can’t extract value out of your data if you aren’t correctly capturing, moving and storing that data.

Unfortunately, this approach has a sinister downside. It creates an isolated bubble of data-related activities and projects. And the longer you stay in the bubble, the harder it is to push beyond that bubble.

Great data projects don’t happen because isolated data scientists are casually strolling through the data, looking for interesting tidbits. The best data scientists I know today build strong bridges between their team and the business team they support. They understand that the data is there to augment the business, and isn’t there as an end-all-be-all panacea.

Algorithms don’t add value, people do

The most accurate forecast model ever created adds no value if the business doesn’t do something with it

A number of years ago, I was working for a company that was struggling. Sales were down, costs were up, and the industry was in the middle of a seismic shift. The company was caught in a bad situation and was unprepared to deal with it. To make matters worse, we had purchased several other struggling competitors as a way to “bring consolidation” to the market, but all it did was increase the number of low-revenue/high cost problems we needed to solve.

My data science team decided it was time to start leveraging our data to create some actionable tools that could start moving us in a better direction. We set to work building a complex algorithm that would create a series of benchmarks around our customer’s buying history, compare each customer against the benchmark, and expose each customer’s unique “hidden opportunities”. We reasoned that mining over 1 Billion records to find nuanced, customer-specific buying patterns would arm our sales team with critical insights about their clients that would help them target their conversations and drive sales. It was a brilliant plan.

How much increased revenue did it generate? None. We tweaked the algorithm for almost 3 months, adding depth, complexity and accuracy. Getting it “just right” was important. Then we gave it to the sales team who… did nothing. It was too abstract, too complicated. They poked holes in calculations. In the end, it never was rolled out to the sales team, and 3 months of great Data Science efforts were wasted.

6-8 years ago, the industry told everyone that Data Science was the answer. It told employees to rush out and re-brand their skills, or train on those skills. Data Science Bootcamps were created, Universities built new Masters programs. Coursera and Udemy exploded with online certifications. Don’t get me wrong, these are good things! Great things, even.

But what was missing from the R and Python training, the Hive, Spark, Tensorflow, and AI training, the Bayesian probabilities and the clustering techniques… was the reason we create these things in the first place. It’s not about the algorithms. It’s about the human behaviors that our data influences. That’s when data becomes valuable. If we’re not creating things for humans, with the goal to influence those humans in some meaningful way, then our fanciest and most sophisticated algorithms won’t add a drop of value.

Knowing what to focus on

Data can solve problems, but knowing which problems to solve is the real question

Erin is the VP of Sales at your organization. Sales are flat and Erin is growing increasingly anxious about getting back to growth. You decide to take a look at the customer sales data. Is the miss coming from a certain product line? Is it a customer retention problem? Are new business efforts lower than expected? Perhaps deal sizes are shrinking? After some digging, you find that most of your sales are doing fine, but the annual revenue generated by 8 out of your top 10 customers has shrunk over the past 12 months. Erin is now armed with just the information she needs to address the problem and get the company moving in the right direction again.

But what if Sales are booming? Erin is likely just trying to stay above water with all the new deals. She has plenty of problems, but they’re quite different. Now she’s worried about how the Operations team is going to handle the influx of new orders. She needs to make sure that legal has time to review all the contracts. She’s looking into hiring new sales staff to keep up with demand. If you come to Erin with a customer analysis and breakdown of where she’s missing, it probably won’t be received well. She’s got too many problems already and doesn’t have time to fix this problem too. Erin doesn’t need NEW problems, she needs to solve NOW problems.

The point is, you could do the same analysis in both situations, and receive a drastically different response from the leader. That’s because data is only useful when it is being used to solve problems that your stakeholders care about. When there’s misalignment between the business and the data teams, you miss huge opportunities to leverage data.

Business teams don’t understand data

Don’t turn your marketing team into Analysts… arm them with data to become even better at their jobs

Do you have that special person in your life that’s still slightly scared of their computer? Like it’s going to eat them if they turn it on? This is exactly how my grandmother feels. We finally got her to log into her email account once a week, but it took 3 years of convincing.

My grandma is a brilliant lady. And yet she still spends an enormous amount of time with basic tasks like maintaining her calendar, managing daily tasks, basic communication. The computer is there to help her, yet she keeps doing things the way she knows. She’s COMFORTABLE with her old process. She knows it, and it hasn’t failed her so far.

I would argue that most business lines have a similar comfortability with their own industry knowledge. It’s what has gotten them to this point in their careers, and they’ll be damned if a data person comes in here and tells them something different. It’s not that they think data is bad, or doesn’t add value. In fact, if you asked them, I bet 9 out of 10 would say that they need their business line to be more analytical.

But saying you should do something, and actually doing it are very different things. What is stopping them from taking a data-informed approach? It’s fear of what they don’t know. Using data gives up control and safety of their industry expertise for something foreign, confusing, and different.

The lesson here is that getting your organization to use data isn’t about better algorithms, more hadoop clusters, or even more dashboards. It’s about the business becoming comfortable with using the data they have and blending it seamlessly with what they already know.

Executives haven’t truly bought in

executives need to live it the data-driven mindset, building it into their plans, and bringing their teams along

You’d be hard pressed to find an executive that hasn’t added data and data science to their list of yearly goals. They’ve read enough books, seen enough articles, and heard their colleagues and competitors talk about it to know that it can help them. So they dip their toes in the water… perhaps a bullet point at the end of their annual roadmap deck that says “oh yeah, and one more thing… we need to be data-driven”.

Wow a whole bullet point!? What a show of commitment! I’m sure your teams will jump right on it. I mean, they have no training in this, they don’t really know what “data-driven” even means, and they likely have a 10 other areas of focus coming out of that roadmap deck. But yeah, I’m sure it will get done.

Joking aside, most executives have likely allocated a bit of resources, but they’re not fully committed to the idea. To them, it’s still a nice thought experiment. But this is exactly the problem. It signals to their teams that they don’t need to take data seriously. A passing fad.

The unfortunate side-effect of leaders who keep data at an arm’s length, is that when the going gets rough, they are unlikely to rely on it in that critical moment. And they’re even less likely to ensure their teams rely on it. They’ll revert back to their guts. There will be “reasons” why we can’t rely on the numbers, especially when the numbers aren’t stellar. You will hear things like “You didn’t consider that our biggest customer doesn’t go through that [data collection] system”. Or, perhaps “we already tried analyzing that data and it was inconclusive”. Or, the worst “we already knew this information”.

The point is that an organization must have leadership on board with a data-informed culture. They can make or break whether the organization captures the value it’s seeking. Make sure your leaders are actually on the train, not just punching a ticket.

So What is Data Culture?

What is the solution to Data Science’s woes? A more HUMAN approach to data. One where the focus is on alignment between the executive teams, the business teams, and the data teams.

You want to be successful with data?

Do the hard work to ingrain it into the core DNA of your organization.

It needs to permeate how each employee thinks, that they have a voice in their heads asking “what does the data tell us about this?”.

When the business has a fundamental understanding of data, it allows them to speak a common language, often referred to as Data Literacy. This common language builds trust and encourages collaboration between the business team and the data scientists. It opens up the opportunity for them to ask bigger, more impactful questions because they know that they can even attempt to ask them. It ensures that they are more comfortable with a sophisticated mathematical solution to their problem, even if they don’t fully understand it.

But most importantly, it allows the business to bring data to the table, combine it with their deep domain expertise, and make an EVEN BETTER decision than they would have otherwise.

Want to use data more effectively? Align your data science teams with leaders and business teams to make sure they’re all moving in the same direction, and basically aware of each other’s needs and capabilities.

Create a culture of data, help your business team “speak the language of data”, and make sure the data team is tightly aligned with executive & business team objectives. Do this, and you’re all but guaranteed to see data project success rates well above 20%.

---

---