Martin -
Lots of things to check
Ensuring your fuel pumps and pressure regulator are all in order, start with the float levels and ensure they are correct.
Another thing to check is the spark plugs, if the carb is jetted incorrectly your plugs will be the best way to tell that.
Listen for vaccum leaks, and check you base gasket to ensure that it is correct as well as all unused vacuum lines on the carb are plugged. A water spray bottle can help as the engine speed will change as you spray a mist into anything sucking air.
Don't know what you had on before but if it didn't have the same problem you can at least be sure it's not something else.
You didn't mention if the carb was brand new or a used one, which could also be an internal problem to the carb.
For the starting off stumble you might need to check the primary accelerator pump and related settings. If a used carb most people mess with accel pumps (Always on the more side) and the jets. You should be able to find some online help in setting up this, or pick up one of the holley tuning books to set things stock. Most things are pretty simple unless someone butcherd the carb and sold it off, I have one on my Mustang that I got off EBAY and I'll likely have to have it gone through as I can't quite get it right with my holley hacking skills
BTW What size carb and motor are you running? Mechanical or Vacuum Secondaries?
Sandy