It depends on the season. Too high temperature creates turbulences that limit the realization of the flights. But, from May to September approximately, flights can only be done in the morning or late afternoon.
The rest of the time, we can consider, in general, a flight at any time of the day if weather conditions and temperature are mild.
The balloon flights can take place all year round if the weather permit it. Each season with its colors and smells will bring different sensation.
Do not apprehend the cold on board, most of the time, it is warmer during the flight than on the ground. Even in winter !