What you would really need is a bearing that is designed for axial loads as well as radial. There are deep groove bearings that can take positive and negative loads but the difference between the ID and OD is usually larger or you could use a bearing that has axial load is a single direction, such as an angular contact bearing and put one on the outside of the wheel and reverse the bearing on the inner side.
Bearings use a standard called the Angular Bearing Engineers Committee or ABEC rating. Normal cheap bearings are ABEC-5 but you can get 7 and 9 with 9 being the highest precision, also the most expensive. ABEC-5 bearings are good because they have the least friction from new but wear out quicker
Cheers