Starting from ASCII character 0 (the NULL character), all the way to ASCII character 127, Excel sorts all the characters from character 0 to charaxxter 31, and character 127, first. Then, it sorts all the punctuation marks and other non-numeric and non-alphabetic charaxters, then, all the numerics (the ten digits, 0, 1, 2...9, and, finally, the 26 upper-case and the 26 lower-case letters of the alphabet. The letters are sorted in a non-case sensitive mode. For instance, if you sort A,B,a,c, you will get A,a,B,b. And, if you sort a,A,b,B, you will get a,A,b,B.
In closing, I don't know why Microsoft chose to go against the conventional method of sorting, stricktly in accordance with the ASCII code, and it was painful for me to find out this fact, through the help of others. So, I want to share this knowledge here. And, if anybody is really interested in doing the actual sort, send me a PM, include your email address, and I will get in touch with you on this.