When VCHA is enabled, a new OS user “vcha” is created and used for the file based replication.
This morning I experienced an issue on one of my VCHA setups (but not the other) where replication was failing when I arrived into the office.
Update 23/01/2017 – This is a real issue and not a once off. VMware Engineering are aware.
Looking at my vcha.log on my active I could see it was complaining about a password expiration.
2017-01-16T08:40:33.725Z error vcha[7FC6BE0E6700] [Originator@6876 sub=VchaUtil] Error executing command /usr/bin/rsync: exit status=, stdout=, stderr=[ --> VMware vCenter Server Appliance 126.96.36.19900 --> --> Type: vCenter Server with an external Platform Services Controller --> --> WARNING: Your password has expired. --> Password change required but no TTY available. --> rsync: connection unexpectedly closed (0 bytes received so far) [sender] --> rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.2] --> ] 2017-01-16T08:40:33.725Z warning vcha[7FC6BE0E6700] [Originator@6876 sub=RsyncRepl-largeFrp] Rsync failed for vmw, retrying in 8 secs
Using the command chage I inspected the vcha user
On the broken setup it showed that the password expired on Jan 13th 2017
root [ / ]# chage --list vcha Last password change : Nov 14, 2016 Password expires : Jan 13, 2017 Password inactive : never Account expires : never Minimum number of days between password change : 1 Maximum number of days between password change : 60 Number of days of warning before password expires : 7
Looking at my second VC the exact same is present except it looks like the password automatically changed on Jan 13.
root [ / ]# chage --list vcha Last password change : Jan 13, 2017 Password expires : Mar 14, 2017 Password inactive : never Account expires : never Minimum number of days between password change : 1 Maximum number of days between password change : 60 Number of days of warning before password expires : 7
Both VCs were deployed on the same date – Nov 14th 2016. I’m not sure why one automatically changed it’s password but the other didn’t. Also the chage command shows that the account is never supposed to expire so I’m not sure there either.
To fix this issue I manually reset the vcha user password by running:
Set the user account vcha to never expire by running:
chage -m 0 -M 99999 vcha
You need to do this on all three VCHA nodes and reset each node with the same password.
Since Jan 14th was 60 days after the GA of vSphere 6.5 if you hit this issue please file an SR with VMware Support – hopefully this was just a once-off in my environment.