There are incredible bows of both sorts that do equally the same thing. Much of the feel and sound is subjective and in ones head but to compare two bows you would need the exact same wood from the same section of the same tree treated the same way.......
I believe I have tested more round bows that I prefer, allowing possibly more freedom and flexibility than I have octagonal bows. Bows usually start out octangular because this is the way they are cut to begin with. At any given point a maker will decide whether or not he wants to round it based on it's hardness/flexibility. This is not to say that a octangular bow will not be as good as a round one by any means.
Octangular bows tend to be stiffer.