Is scrum master involved at early stage of project like in POC of project

A Scrum Master has no role in the progress of the project.  The Scrum Master has a role in creating an environment and state of mind within the organization that enables teams to organize and manage themselves to deliver products.  A Scrum Master has the responsibility to the Scrum Team to help them and the wider organization to understand the Scrum framework and how everyone can best interact/contribute to the success of the Scrum Teams.

You are a member of the Scrum Team but you are not responsible for the development activities that are undertaken.  Those activities are solely the responsibility of the Developers.  You are not responsible for the direction that the product takes. That is solely the responsibility of the Product Owner. You are responsible for the success and understanding of the Scrum framework in the organization and as it relates to the success of your Scrum Team.

Now, I will admit that this is all based upon the roles and responsibilities defined in the Scrum Guide. Pay notice to the fact that I said "roles and responsibilities" not job title or job description.  Scrum Master is not defined as a job title in the Scrum Guide.  Your organization most likely does have a job description for Scrum Master that applies to your duties.  It is up to you to decipher that.  My answer is entirely based upon the Scrum framework definition from the Scrum Guide.

What do people hope to accomplish via this so-called POC? In Scrum, isn't each Sprint an experiment, and a small project in the application of validated learning?

Currently I am facing issue that my team is not much ready to involved scrum master as they are working on POC and nothing is concrete to start

You may need to explain to your team that Scrum doesn't require things to be very much 'concrete' to start.

It is enough to have some idea about the Product Goal. It sounds like your Product Owner already has the Product Goal in mind since they decided to do the POC. 

So, you are perfectly fine to start sprinting.

Creating the POC might be a good Sprint Goal for your 1st Sprint. If it's too much to digest in one Sprint, they may break it into some interim Sprint Goals. 

The Scrum Master can be involved in the early stages of a project, including the POC phase. Their role includes facilitating collaboration, guiding Agile practices, supporting development processes, removing impediments, and promoting continuous improvement. Involving the Scrum Master can help the team progress and overcome challenges during the POC.