Evaluation of Team performance

Story points are of no use to you: they only help the Developers get their arms around how much work they can take on each Sprint to meet a Sprint Goal.

Are those Sprint Goals being met, and are Done increments of immediate usable quality being provided each and every Sprint? Scrum is commitment driven.

If the Developers are not meeting these commitments, find out why. Perhaps there are organizational impediments holding them back.

@Ian's suggestion is spot on. Who really cares if a team burns story points if there is nothing being delivered that the stakeholders want and need at a time they want and need it. 

Story points are an estimate at a specific point in time based upon the information available at that time. As soon as work begins, those estimates are no longer valid because new information has been obtained. Even if you were to use flow metrics (I recommend you read these books to learn about them - https://actionableagile.com/books/) you still know nothing about the true performance of the team.  I am assuming that the performance that the organization wants is something the makes money.  If you aren't delivering value that the stakeholders/customers/users want and need, then you won't be making money. Stop trying to put the emphasis on whether the development team is busy and start putting the emphasis on whether the stakeholders/customer/users are happy.

I also think Ian's suggestion is spot on here.

I'd add that, regardless of what you do, story points are the wrong measure. Consider that story points are initial estimates of the work based on what is known at the time. Unless you invest the time to update them after the work is completed (which I wouldn't recommend, as there are better uses of the team's time), they do not accurately reflect the actual time or effort required to complete the work. At best, story points are an initial planning tool, and I'm not even convinced that they are really useful for that for many teams.

I think it's key to understand what "doing well" means. Tracking whether the team achieved the Sprint Goal or not is one option that inherently comes from the Scrum framework. However, even that doesn't say anything about if the Sprint Goal was valuable enough to warrant the cost of running a Sprint - this information may not be available until after the Sprint is complete, though. There may be other options, like understanding where the team is spending their time and maximizing the time on value-add activities. However, these options aren't universally defined and are likely ot be unique to each team's context.

I wanted to add another resource for you to read.  It occurred to me that it might be helpful for you as well.  

Evidence-Based Management might be a useful read for you. It helped me understand how to discuss the topic of evaluating the organizations ability to be agile with more than one executive. 

https://www.scrum.org/resources/evidence-based-management

Thank you for your responses. Yes, I understand that story points are not an appropriate metric for measuring team performance. However, I need to show the company that our time-to-market is improving and that our Sprint Goals are appropriately scoped and consistently achieved (without over- or under-committing), along with similar indicators. So, again, what metrics do you use to evaluate whether your team is performing well, and how do you measure them?

There is great advice in the replies above - focus on outcomes over outputs.

Consider exploring flow metrics to see how your delivery system is performing. Ian provided a link to some amazing books above. 

Cycle time is the elapsed time from when work starts and when it finishes. Shorter cycle time usually means faster time to market. You can track this over time using percentiles on cycle time scatter plot (not as an average), choosing which percentile is most appropriate for your context. 

Throughput is the count of finished items over a period of time. Finished items presumably represent something of value delivered to customers. You can measure to see if this is increasing over time.

With these and any metrics, you will want to pair them with outcomes and results. Higher throughput only matters if what’s being completed delivers value and achieves the outcomes you’re aiming for.

If you want to measure time-to-market, specifically, there are plenty of options.

I'd start with the DORA metrics. The four key metrics - lead time, deployment frequency, change fail percentage, and failed deployment recovery time - all contribute to time-to-market. The most important would probably be the lead time and deployment frequency, but frequent deployments that often fail and/or take a long time to recover from aren't very effective.

I've found that the DORA definition of "lead time" is closer to what I would consider "cycle time". You may want to consider measuring time between different points in the workflow, from initial submission through development starting through integration through deployment. Without a deep understanding of your workflow, I can't give suggestions of which points may be interesting to observe.

Tracking a binary measure of Sprint Goal achieved for each Sprint would also be useful. At the Sprint Review, assess the work done by the team and determine if they (and the stakeholders) believe that the Sprint Goal was satisfies or not. If not, the Sprint Retrospective would be a good place to dive into why the goal wasn't met and what can be done to improve in the future.

I'd keep in mind that time-to-market is one specific type of performance, though. EBM also discusses the team or organization's Ability to Innovation as another set of capability measures. Lean gives the concept of waste and non-value-adding work, which can be measured and reduced to improve performance.

I go back to my first post as an answer. Add to it @Ryan's and @Thomas' posts and you have good beginning. 

One thing I want to mention about using the Sprint Goal as a basis.  If the Sprint Goal is as described in the Scrum Guide, using it as a metric is good.  Here is what the Scrum Guide says about the Sprint Goal

The Product Owner proposes how the product could increase its value and utility in the current Sprint. The whole Scrum Team then collaborates to define a Sprint Goal that communicates why the Sprint is valuable to stakeholders. 

If your Sprint Goals are always defined as "complete all of the items in the Sprint Backlog", then it isn't a good value metric to track achievement of the Sprint Goal.  If the Sprint Goals are similar to "create the ability for users to update their inventory by importing shipping manifests", they can be better at evaluating value. 

A Sprint Goal might be based upon flow, such as improved cycle time or throughput or any of the DORA metrics.

The ability to shift between feature-based Sprint Goals and flow-based ones, or any other coherence that makes the Sprint valuable to stakeholders, is useful when facing complex and inconsistent challenges.

A Sprint Goal might be based upon flow, such as improved cycle time or throughput or any of the DORA metrics.

The ability to shift between feature-based Sprint Goals and flow-based ones, or any other coherence that makes the Sprint valuable to stakeholders, is useful when facing complex and inconsistent challenges.

Ian's comment here is significant, and I think it's something that many people overlook. In my experience, people tend to want to force the Sprint Goal to be about the deliverable product or service. Nothing says that it needs to be. As long as the Sprint Goal provides a "single objective for the Sprint" and "creates coherence and focus, encouraging the Scrum Team to work together", it very well could be a process-oriented goal. There could be valid Sprint Goals around increasing throughput, decreasing cycle time, or preventive work to make deployment failures/rollbacks less likely. The work needed to do these may not be visible to stakeholders in the end product or service, but that doesn't mean that the work isn't valuable.

@Ian, @Thomas.

Thanks for that input. I have never thought it that way.  I learned something today.