The first digital computer (according to some) the Z3, built by Konrad Zuse in 1941 used floating point representation. This computer had a clock speed of five to ten Hertz and a memory of 64 words of 22 bits! (The earlier Z1 machine also used floating point, but was not quite programmable enough to be considered a computer.)
Early computers were built to do engineering and scientific calculation so it is no surprise that the invention of floating point happened at the same time. In later years it was realized that computers are very useful things, and not just for calculation. Often floating point was not supported.
Many early minicomputers and microprocessors did not directly support floating point in hardware. For example, Intel processor chips before the 80486 did not directly support floating point. A floating point operation in a program (written in C, say) was compiled into a sequence of bit-manipulation instructions that did the required operations. Computers used for graphics or engineering calculation often had an additional chip that performed floating point operations in hardware.
MIPS processors are very strong on floating point, and have supported the IEEE standard from their (and its) beginning. MIPS chips are often used in high-end engineering and graphics workstations and are famous for their fast floating point.
(Not very hard thought question: ) Do you imagine that in (say) 1975 that the floating point methods used on IBM mainframes were the same as on DEC minicomputers?