These men have "gone to the mattresses".
Basically, going to war with a rival family.
Its when a mafia family sends someone out to rent apartments and some mattresses for the soldiers of the family to sleep on while they hide out in safety, waiting for a call to action.
Obviously, there's a lot of waiting around, so card games, cooking, playing or listening to music would take up most of the time waiting for a phone call.
The novel explains further...
Whenever a war between the Families became bitterly intense, the opponents would set up headquarters in secret apartments where the “soldiers” could sleep on mattresses scattered through the rooms. This was not so much to keep their families out of danger, their wives and little children, since any attack on noncombatants was undreamed of. All parties were too vulnerable to similar retaliation. But it was always smarter to live in some secret place where your everyday movements could not be charted either by your opponents or by some police who might arbitrarily decide to meddle.
And so usually a trusted caporegime would be sent out to rent a secret apartment and fill it with mattresses. That apartment would be used as a sally port into the city when an offensive was mounted
The Godfather - Mario Puzo