Run engine, turn it off, remove fuel pump fuse and start after 3-5 min.
If everything is OK, fuel system would still have pressure enough to start engine.
You can find out how bad regulator is. If engine doesn't start after 10-20 sec, it is bad.

 

Ok, let's say the engine does not start after 20 seconds.

What's the easiest way to decide whether it's the fuel pump or the regulator?

 

It will have enough pressure for a chug of fumes and some. That does not tell you much at all.
Its not direct Injection by any means after two revolutions of crank its pretty much depleted.
You check a regulator on a return fuel system by looking at your fuel trim and o2's
The days of carbs and single throttle body injection are long gone partna!