Little is known of his background except that he was a Msotho and became a qualified bookkeeper. In the early 1920s, he became financial secretary of the Industrial and Commercial Workers' Union (ICU), based in Cape Town, and also was recruited into the Communist Party of South Africa (CPSA).
In late 1926 he was expelled from the ICU when its national conference voted to exclude Communists from the executive committee. In 1927 he became one of the first Africans elected to the central committee of the CPSA. When J. T. Gumede became president-general of the African National Congress (ANC) in June 1927, Khaile was elected ANC general secretary.
Gerhart G.M and Karis T. (ed)(1977). From Protest to challenge: A documentary History of African Politics in South Africa: 1882-1964, Vol.4 Political Profiles 1882 1964. Hoover Institution Pres: Stanford University.