Question : sendmail - stat=Deferred: Connection refused by [127.0.0.1]

We have recently started seeing messages like the following in our /var/log/maillog.

Sep 27 10:48:25 www3 sendmail[823]: k8RHmOvL000823: to=Doug ngular.com>, ctladdr=xamine (501/501), delay=00:00:01, xdelay=00:00:00, mailer=relay, pri=30482, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection refused by [127.0.0.1]

We are running Red Hat Enterprise Linux ES release 4 (Nahant Update 3) with Sendmail version 8.13.1

What is causing sendmail to reject connections to localhost? I bumped up the value of MaxQueueChildren and this seems to have helped a bit but hasn't completely eliminated the problem. Does sendmail normally connect to localhost first to send e-mail addresses externally as in the log snippet above? I am certainly not a sendmail expert, but it seems strange that it would connect to localhost to do this.

The problem seems to be intermittent, but bursty in that we won't see these for awhile then all of a sudden we get several and the problem goes away. I have scoured the Net searching for similar issues and have tried lots of things, but most recently the following:

I can provide additional details as necessary but want to get this posted as I am hoping there will be someone out there who has seen this exact problem and have a good solution to try.

Thanks,

1) Modifying the /etc/mail/access file and re-generating (of course restarting sendmail after the changes) it to include the following settings:

--------------------------------------------------
localhost.localdomain           RELAY
localhost                       RELAY
127.0.0.1                       RELAY
XX.XXX.XXX.XXX                  RELAY
# Don't delay for internal hosts
GreetPause:localhost            0
# Do not limit number of connections for localhost
ClientRate:127.0.0.1            0
# Do no limit number of simultaneous connections from localhost
ClientConn:127.0.0.1            0
-----------------------------------------

2) Editing the sendmail.cf file (I know, I know, one should always modify the sendmail.mc file as per the sendmail Performance Tuning Guide - but I am taking some shortcuts now to resolve this problem ASAP) as follows and restarting sendmail:

--------START sendmail.cf snippet---------------------------------------------------------------
# level 10 config file format
V10/Berkeley

# override file safeties - setting this option compromises system security,
# addressing the actual file configuration problem is preferred
# need to set this before any file actions are encountered in the cf file
#O DontBlameSendmail=safe

# default LDAP map specification
# need to set this now before any LDAP maps are defined
#O LDAPDefaultSpec=-h localhost

##################
#   local info   #
##################

# my LDAP cluster
# need to set this before any LDAP lookups are done (including classes)
#D{sendmailMTACluster}$m

Cwlocalhost
# file containing names of hosts for which we receive email
Fw/etc/mail/local-host-names

# my official domain name
# ... define this only if sendmail cannot automatically determine your domain
#Dj$w.Foo.COM

# host/domain names ending with a token in class P are canonical
CP.

# "Smart" relay host (may be null)
DS


# operators that cannot be in local usernames (i.e., network indicators)
CO @ % !

# a class with just dot (for identifying canonical names)
C..

# a class with just a left bracket (for identifying domain literals)
C[[

# access_db acceptance class
C{Accept}OK RELAY


C{ResOk}OKR


# Hosts for which relaying is permitted ($=R)
FR-o /etc/mail/relay-domains

# arithmetic map
Karith arith
# macro storage map
Kmacro macro
# possible values for TLS_connection in access map
C{Tls}VERIFY ENCR





# dequoting map
Kdequote dequote

# class E: names that should be exposed as from this host, even if we masquerade
# class L: names that should be delivered locally, even if we have a relay
# class M: domains that should be converted to $M
# class N: domains that should not be converted to $M
#CL root
C{E}root
C{w}localhost.localdomain



# my name for error messages
DnMAILER-DAEMON


# Mailer table (overriding domains)
Kmailertable hash -o /etc/mail/mailertable.db

# Virtual user table (maps incoming users)
Kvirtuser hash -o /etc/mail/virtusertable.db

CPREDIRECT

# Access list database (for spam stomping)
Kaccess hash -T -o /etc/mail/access.db

# Configuration version number
DZ8.13.1


###############
#   Options   #
###############

# strip message body to 7 bits on input?
O SevenBitInput=False

# 8-bit data handling
#O EightBitMode=pass8

# wait for alias file rebuild (default units: minutes)
O AliasWait=10

# location of alias file
O AliasFile=/etc/aliases

# minimum number of free blocks on filesystem
O MinFreeBlocks=100

# maximum message size
#O MaxMessageSize=0

# substitution for space (blank) characters
O BlankSub=.

# avoid connecting to "expensive" mailers on initial submission?
O HoldExpensive=False

# checkpoint queue runs after every N successful deliveries
#O CheckpointInterval=10

# default delivery mode
O DeliveryMode=background

# error message header/file
#O ErrorHeader=/etc/mail/error-header

# error mode
#O ErrorMode=print

# save Unix-style "From_" lines at top of header?
#O SaveFromLine=False

# queue file mode (qf files)
#O QueueFileMode=0600

# temporary file mode
O TempFileMode=0600

# match recipients against GECOS field?
#O MatchGECOS=False

# maximum hop count
#O MaxHopCount=25

# location of help file
O HelpFile=/etc/mail/helpfile

# ignore dots as terminators in incoming messages?
#O IgnoreDots=False

# name resolver options
#O ResolverOptions=+AAONLY

# deliver MIME-encapsulated error messages?
O SendMimeErrors=True

# Forward file search path
O ForwardPath=$z/.forward.$w:$z/.forward

# open connection cache size
O ConnectionCacheSize=2

# open connection cache timeout
O ConnectionCacheTimeout=5m

# persistent host status directory
#O HostStatusDirectory=.hoststat

# single thread deliveries (requires HostStatusDirectory)?
#O SingleThreadDelivery=False

# use Errors-To: header?
O UseErrorsTo=False

# log level
O LogLevel=9

# send to me too, even in an alias expansion?
#O MeToo=True

# verify RHS in newaliases?
O CheckAliases=False

# default messages to old style headers if no special punctuation?
O OldStyleHeaders=True

# SMTP daemon options

O DaemonPortOptions=Port=smtp,Addr=127.0.0.1, Name=MTA

# SMTP client options
#O ClientPortOptions=Family=inet, Address=0.0.0.0

# Modifiers to define {daemon_flags} for direct submissions
#O DirectSubmissionModifiers

# Use as mail submission program? See sendmail/SECURITY
#O UseMSP

# privacy flags
O PrivacyOptions=authwarnings,novrfy,noexpn,restrictqrun

# who (if anyone) should get extra copies of error messages
#O PostmasterCopy=Postmaster

# slope of queue-only function
#O QueueFactor=600000

# limit on number of concurrent queue runners
O MaxQueueChildren=12

# maximum number of queue-runners per queue-grouping with multiple queues
#O MaxRunnersPerQueue=1

# priority of queue runners (nice(3))
#O NiceQueueRun

# shall we sort the queue by hostname first?
#O QueueSortOrder=priority

# minimum time in queue before retry
#O MinQueueAge=30m

# how many jobs can you process in the queue?
#O MaxQueueRunSize=10000

# perform initial split of envelope without checking MX records
#O FastSplit=1

# queue directory
O QueueDirectory=/var/spool/mqueue

# key for shared memory; 0 to turn off
#O SharedMemoryKey=0



# timeouts (many of these)
#O Timeout.initial=5m
O Timeout.connect=1m
#O Timeout.aconnect=0s
#O Timeout.iconnect=5m
#O Timeout.helo=5m
#O Timeout.mail=10m
#O Timeout.rcpt=1h
#O Timeout.datainit=5m
#O Timeout.datablock=1h
#O Timeout.datafinal=1h
#O Timeout.rset=5m
#O Timeout.quit=2m
#O Timeout.misc=2m
#O Timeout.command=1h
O Timeout.ident=0
#O Timeout.fileopen=60s
#O Timeout.control=2m
O Timeout.queuereturn=1d
#O Timeout.queuereturn.normal=5d
#O Timeout.queuereturn.urgent=2d
#O Timeout.queuereturn.non-urgent=7d
#O Timeout.queuereturn.dsn=5d
O Timeout.queuewarn=4h
#O Timeout.queuewarn.normal=4h
#O Timeout.queuewarn.urgent=1h
#O Timeout.queuewarn.non-urgent=12h
#O Timeout.queuewarn.dsn=4h
#O Timeout.hoststatus=30m
#O Timeout.resolver.retrans=5s
#O Timeout.resolver.retrans.first=5s
#O Timeout.resolver.retrans.normal=5s
#O Timeout.resolver.retry=4
#O Timeout.resolver.retry.first=4
#O Timeout.resolver.retry.normal=4
#O Timeout.lhlo=2m
#O Timeout.auth=10m
#O Timeout.starttls=1h

# time for DeliverBy; extension disabled if less than 0
#O DeliverByMin=0

# should we not prune routes in route-addr syntax addresses?
#O DontPruneRoutes=False

# queue up everything before forking?
O SuperSafe=True

# status file
O StatusFile=/var/log/mail/statistics

# time zone handling:
#  if undefined, use system default
#  if defined but null, use TZ envariable passed in
#  if defined and non-null, use that info
#O TimeZoneSpec=

# default UID (can be username or userid:groupid)
O DefaultUser=8:12

# list of locations of user database file (null means no lookup)
O UserDatabaseSpec=/etc/mail/userdb.db

# fallback MX host
#O FallbackMXhost=fall.back.host.net

# fallback smart host
#O FallbackSmartHost=fall.back.host.net

# if we are the best MX host for a site, try it directly instead of config err
O TryNullMXList=true

# load average at which we just queue messages
O QueueLA=40

# load average at which we refuse connections
O RefuseLA=50

# log interval when refusing connections for this long
#O RejectLogInterval=3h

# load average at which we delay connections; 0 means no limit
#O DelayLA=0

# maximum number of children we allow at one time
O MaxDaemonChildren=6

# maximum number of new connections per second
O ConnectionRateThrottle=100

# Width of the window
#O ConnectionRateWindowSize=60s

# work recipient factor
#O RecipientFactor=30000

# deliver each queued job in a separate process?
#O ForkEachJob=False

# work class factor
#O ClassFactor=1800

# work time factor
#O RetryFactor=90000

# default character set
#O DefaultCharSet=iso-8859-1

# service switch file (name hardwired on Solaris, Ultrix, OSF/1, others)
#O ServiceSwitchFile=/etc/mail/service.switch

# hosts file (normally /etc/hosts)
#O HostsFile=/etc/hosts

# dialup line delay on connection failure
#O DialDelay=10s

# action to take if there are no recipients in the message
#O NoRecipientAction=add-to-undisclosed

# chrooted environment for writing to files
#O SafeFileEnvironment=/arch

# are colons OK in addresses?
#O ColonOkInAddr=True

# shall I avoid expanding CNAMEs (violates protocols)?
#O DontExpandCnames=False

# SMTP initial login message (old $e macro)
O SmtpGreetingMessage=$j Sendmail $v/$Z; $b

# UNIX initial From header format (old $l macro)
O UnixFromLine=From $g $d

# From: lines that have embedded newlines are unwrapped onto one line
#O SingleLineFromHeader=False

# Allow HELO SMTP command that does not include a host name
#O AllowBogusHELO=False

# Characters to be quoted in a full name phrase (@,;:\()[] are automatic)
#O MustQuoteChars=.

# delimiter (operator) characters (old $o macro)
O OperatorChars=.:%@!^/[]+

# shall I avoid calling initgroups(3) because of high NIS costs?
#O DontInitGroups=False

# are group-writable :include: and .forward files (un)trustworthy?
# True (the default) means they are not trustworthy.
#O UnsafeGroupWrites=True


# where do errors that occur when sending errors get sent?
#O DoubleBounceAddress=postmaster

# where to save bounces if all else fails
#O DeadLetterDrop=/var/tmp/dead.letter

# what user id do we assume for the majority of the processing?
#O RunAsUser=sendmail

# maximum number of recipients per SMTP envelope
#O MaxRecipientsPerMessage=0

# limit the rate recipients per SMTP envelope are accepted
# once the threshold number of recipients have been rejected
#O BadRcptThrottle=0

# shall we get local names from our installed interfaces?
O DontProbeInterfaces=true

# Return-Receipt-To: header implies DSN request
#O RrtImpliesDsn=False

# override connection address (for testing)
#O ConnectOnlyTo=0.0.0.0

# Trusted user for file ownership and starting the daemon
#O TrustedUser=root

# Control socket for daemon management
#O ControlSocketName=/var/spool/mqueue/.control

# Maximum MIME header length to protect MUAs
#O MaxMimeHeaderLength=0/0

# Maximum length of the sum of all headers
#O MaxHeadersLength=32768

# Maximum depth of alias recursion
#O MaxAliasRecursion=10

# location of pid file
#O PidFile=/var/run/sendmail.pid

# Prefix string for the process title shown on 'ps' listings
#O ProcessTitlePrefix=prefix

# Data file (df) memory-buffer file maximum size
#O DataFileBufferSize=4096

# Transcript file (xf) memory-buffer file maximum size
#O XscriptFileBufferSize=4096

# lookup type to find information about local mailboxes
#O MailboxDatabase=pw

# override compile time flag REQUIRES_DIR_FSYNC
#O RequiresDirfsync=true

# list of authentication mechanisms
#O AuthMechanisms=EXTERNAL GSSAPI KERBEROS_V4 DIGEST-MD5 CRAM-MD5

# Authentication realm
#O AuthRealm

# default authentication information for outgoing connections
#O DefaultAuthInfo=/etc/mail/default-auth-info

# SMTP AUTH flags
O AuthOptions=A

# SMTP AUTH maximum encryption strength
#O AuthMaxBits

# SMTP STARTTLS server options
#O TLSSrvOptions

# Input mail filters
#O InputMailFilters


# CA directory
#O CACertPath
# CA file
#O CACertFile
# Server Cert
#O ServerCertFile
# Server private key
#O ServerKeyFile
# Client Cert
#O ClientCertFile
# Client private key
#O ClientKeyFile
# File containing certificate revocation lists
#O CRLFile
# DHParameters (only required if DSA/DH is used)
#O DHParameters
# Random data source (required for systems without /dev/urandom under OpenSSL)
#O RandFile

############################
# QUEUE GROUP DEFINITIONS  #
############################


###########################
#   Message precedences   #
###########################

Pfirst-class=0
Pspecial-delivery=100
Plist=-30
Pbulk=-60
Pjunk=-100

#####################
#   Trusted users   #
#####################

# this is equivalent to setting class "t"
Ft/etc/mail/trusted-users
Troot
Tdaemon
Tuucp

#########################
#   Format of headers   #
#########################

----------------END sendmail.cf snippet-----------------------------------------------------

Answer : sendmail - stat=Deferred: Connection refused by [127.0.0.1]

"My understanding is that if sendmail is running locally on the box and sending mail, I don't need to have SMART_HOST enabled"

One has absolutely nothing to do with the other. A SMART_HOST is needed if the sendmail server is unable to resolve the correct destination host  for a non-local E-Mail. Is there any reason that the sendmail host might have suddenly become unable to perform DNS lookups or otherwise unable to resolve MX records?