Time does not matter to the Smtp object, so this is most likely a server issue. Could you please create a communication log showing both successful and unsuccessful attempts and either post them here or mail them to us for analysis? That should make it possible to tell why the server is rejecting the message.
Also, please note that the array returned by SmtpException.GetRejectedRecipients() contains a list of rejected addresses along with SMTP server responses, and those are likely to contain additional information as well (the same as the log).