Morning: eat early, then go to the sand
Start at The Cottage for breakfast. The patio is the reason to go, and with 68°F air and only 5 mph of wind from the east, you can sit outside without a jacket. Go early on a Tuesday and you will not need to plan around a wait.
From there, head to La Jolla Shores Beach. The water is 74.3°F, which is a real draw for mid-November, and the surf is 3.3 feet at 12-second intervals. That is enough swell to respect, so wade in at the Shores, where the beach is wide and forgiving. If you want a place to spread out, Kellogg Park sits just steps from the surf and works well for a break between swims.
Midday: the Cove, then lunch with a view
Drive or ride over to La Jolla Cove while the sun is high and the sky is mostly clear. Come to look, not to climb. With a 12-second swell running, stay off the wet rocks at the edge and watch the sea lions from the path instead.
Walk up to Brockton Villa for lunch. The deck looks down on the Cove, so you can keep watching the water while you eat. Skip the idea of eating on the grass by the Cove. The deck gives you a table, shade if you want it, and the same view.
Late afternoon: climb Mount Soledad
Give yourself a quiet hour in the middle of the day, then drive up to Mount Soledad Memorial Park about an hour before sunset. Fall light gets low early, and the viewing circle is a good place to see La Jolla spread out below you. With light wind and mostly sunny skies, it should be a comfortable place to stand and wait for the color.
Sunset dinner on Prospect Street
Finish at Duke's La Jolla. It sits on Prospect Street like a porch for the Pacific, and conversations tend to slow down as the sun drops. Book a table ahead if you want to be sure of a seat as the light fades.
The order of the day matters: sand in the morning, Cove and lunch in the middle, Soledad before dinner. Do it in reverse and you will be fighting the low sun and the dinner rush at the same time.