Born in Oldham, Lancashire, Fitton was born to hard-working parents. Oldham was known to be the centre of the world’s cotton-spinning industry where his Mother worked. Due to the harsh conditions of the economy at the time, James Fitton senior became actively interested in politics and would play a prominent role in the Labour movement. This risky decision made Fitton’s father unemployed forcing him to work under an alias in the future.

Fitton developed a passion for drawing when he was hospital bound after a botched mastoid operation. It was suggested that Fitton’s enhanced visual awareness came stemmed from the damage to another of his senses. On completing his education at the Watersheddings Board School, Fitton began his artistic training in evening classes at Manchester Art School under Adolfe Vallette. Fitton worked mainly in prints and developed his technique to later design for the London Transport posters commissioned by Frank Pick. His work is now held in the collections of the Tate, the Herbert Art Gallery and Museum and the Newport Museum and Art Gallery. Fitton married fellow artist Margaret Cook and both their portraits continue to be show with the Nottingham City Museums and Galleries and the National Portrait Gallery.