The response of soil microbial diversity to long-term fertilization is still not well understood in the context of different soil types. The purpose of this research was to reveal the impact of fertilization systems on soil parameters and life activity of the main taxonomic and physiological groups of microorganisms responsible for nitrogen, carbon, and phosphorus transformation. Reported results were obtained in the course of a 55-year-long experiment on fertilization of sod-podzolic soil in a grain-flax-potato crop rotation. Soil sampling was conducted within a 0–20 cm depth in five sites: without fertilizer (C); organic fertilization system, manure (O1FS); mineral fertilization system, NPK (MFS); organic-mineral fertilization system, manure + NPK (O1MFS); and organic-mineral fertilization system, siderate + NPK (O2MFS). Long-term use of various fertilization systems has led to changes in the soil properties. Bacteria dominated the microbial community in all examined areas. Soil fertilization supported bacteria development in all variants, except for MFS, and negatively affected the micromycetes content. A strong relationship between the change of the main soil indicators and the number of microorganisms from the main taxonomic groups was found between the soil pH KCl and the number of micromycetes. The O1FS option had the most beneficial effect on the development of soil nitrifiers and denitrifiers. The O1MFS fertilization system was the most favorable for the development of non-symbiotic anaerobic nitrogen-fixing, cellulose-degrading and phosphate-mobilizing microorganisms. In turn, the least favorable conditions for the development of physiological groups of microorganisms were found in cases of continuous use of mineral fertilizers.