People may have talked about these things before and I would like to share my experience around it. This story is long dated back when I was working on a software product for a manufacturing domain client. The software was to be deployed on tablet to be used by tow motor operators. While moving the bar-coded hardware material from a rack to another they had to update the new location and other details of the moved material. It was a 100% touch-based system and no mouse or keyword involved.
The team got the initial requirements and we started the development, showed the wire frames to customers and got their buy-in. We had intermediate demo sessions with our customers, got feedback and everything went well. Everyone involved including the shop floor head was very excited. The final day came and the software was deployed on the tablets, ready to be used by the tow motor operators.
Hang-on, they could not use it, they were not able to operate the new cool software. But why? As a part of shop-floor security guidelines, operators wear gloves and the size of buttons on the new software screen was so small that choosing and pressing the required button while wearing gloves was not possible. The shop floor was stopped and we had to rollback to old software. There was no other way and we had to redesign all screens. It doesn’t matter under which category you want to put it – functional or non-functional but it was never included in any discussion between the customers and the development team. Some of the learnings we had from this were -
Sometimes even having a software demo with customers and getting feedback doesn’t work, though it is a good practice. The problem you have with a demo session is, who is using and controlling the software – most of the times it would be you and not your customer. You will be demonstrating things which you would like to demo and not what customer may be interested in.
I can relate it to the problems people share following Scrum. They use Sprint Review to demonstrate their working software to customers while keeping the control. My humble request would be to give the control to your customers/end-users, let them play with the software and provide feedback. Use it as an informal collaboration event for inspecting your Increment and not as a status meeting.
Secondly, we understand the difference between customers and end users but most of the time we ignore the end-users, one of the key stakeholders. In our case they were ignored completely.
Finally, it doesn’t matter how much great reviews you are getting on your wire-frames, videos etc. The real test of your product is when it goes to production and end users start using it. The focus should be to go to production as early as you can, anything before it, is just your assumption and hypothesis which will be validated by the end users.
Otherwise, be ready for the surprises.