If the birds were indeed hawks (as oppose to falcons) they would likely be Red-tailed Hawks. There happens to be an active nest 100 yards from the DVP. Another possibly would be Peregrine Falcons which nest on the buildings downtown. The best way to tell them apart is by size. If they surprised you at being large on first glance then they were likely Red-tails, if they simply seemed to be hawk-like and had pointed wing-tips and a fast driving flight then they were likely Peregrines. Of course, I've only named these two birds of prey because they are the most commonly seen raptors in Downtown toronto. There are other possible species, but given your location and description of their flight, I'd put my money down on Red-tailed Hawk.