Meno. Depends on the density of the rock.
Exaggerated case: Rock the size of an aspirin. Weighs 100 lbs. will push the boat deep into the water. When removed from the boat a large volume of boat ( say 20 gallons) will no longer displace water causing a lowering of the water level. This will be only minimally compensated for by an increase in the water level equal to the volume of the rock.
If you enjoy extreme cases here’s a potentially actual (though not practical) case. Take a teaspoon worth of a neutron star. You will need an extremely strong and massive boat to make this work. A teaspoon of a neutron star weighs about a billion tons. That would cause a ginormous lowering of the water level when superman lifts it up from the boat. But it will cause a displacement when thrown into the ocean only equal to its minuscule volume.