Why the submachine guns don’t have rifling? or they do? the rifling will result in a machine pistol?

The only, common, modern, smoothbore firearm I can think of is a shotgun. Handguns, SMGs, and of course full sized rifles all have some rifling in the barrel.

This is one of the things that’s examined when you’re trying to match a bullet to the gun that fired it. The specifics of the rotation are unique-ish to the model of firearm, and any abnormalities in the barrel, that can be detected on the bullet can be used to identify the individual firearm.

A submachine gun is just a fully automatic firearm chambered to fire handgun rounds. A machine pistol is an informal name for very compact SMGs.

If you don’t have a rifled barrel, the bullet will tumble once it’s left the barrel, which will result in a very random bullet spread. You see this in old muskets and other pre-rifle firearms.

With shotguns, because of the shot itself, rifling doesn’t serve much purpose. I think rifled shotguns do exist, for slug loads, though I can’t remember if any ship with rifled barrels standard. (Probably, but I can’t remember.)