Born in 1936 in British Mandatory Palestine, Berger was raised by a Hasidic family in Jerusalem before moving to the US in 1949 where his athletic career took off