The House of Rolf is a line of nobility that Rolf supposedly descends from. He claims that it is well-known and has a storied history, but throughout the game, it is strongly implied that Rolf has fabricated his noble lineage and is in fact a former bandit. If you make him a vassal, he will train Sarranid troops, with him being an improverished noble.
Nevertheless, granting Rolf a fief is not controversial, so it appears that many vassals more or less believe in his claim of noble birth or he didn't become a bandit until after the loss of his heritage.
Also, some bandits don't attack Rolf since he was a bandit.