I have equipment I want to put on this, modems, routers, monitor etc. Can you tell me what size shelf would fit on this?

Any shelf that is designed for a 19in rack would fit it really depends on the depth of the shelf you want based on the equipment you wish to place on it.

is this rack with small round screws or the square ones?

round screw holes are pre-drilled works with the machined screws that come with most rack equipment. 3 holes per numbers section section, make it easy to use a variety of equipment that may not have standard rack spacing

does it support TYAN FA100B7118, SUPERMICRO SBE-820L, SuperServer 6049GP-TRT or not?

It COULD support those if you have center-mount rack kits. Both SuperMicro's rack kit (MCP-290-00057-0N) and Tyan's kit (452T57400001) assume a 4-post rack, not a 2-post. Personally, I use 4-post racks for servers. 2-post racks are better suited for network equipment (IMHO).