Saturday, November 8, 2014

Repair Synology DSM5.1 for use as a Linux backup repository.

After updating my Synology to DSM 5.1-5004, the following morning I was greeted by a rash of error messages from my Veeam B&R 7 backup jobs: "Error: Server does not support diffie-hellman-group1-sha1 for keyexchange"

I logged into the backup host and re-ran the repository resync process, to be greeted by the same error.
Synology DSM 5.1 error
The version of SSH on the Synology was OpenSSH 6.6p2:

As it turns out, this version of SSH doesn't enable the required key exchange protocol by default; luckily, that's an easy edit of the /etc/ssh/sshd_config file. And to play it safe, I added not only the needed Kex parameter, I also added the published defaults.
KexAlgorithms diffie-hellman-group1-sha1,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha1
After restarting SSH in the DSM control panel, then re-scanning the repository, all was not quite fixed:

Back to the manfile for sshd_conf...

The list of supported ciphers is impressive, but rather than add all of them into the list, I thought it would be useful to get a log entry from the daemon itself as it negotiated the connection with the client. Unfortunately, it wasn't clear where it was logging, so it took some trial-and-error with the config settings before I found a useful set of parameters:
SyslogFacility USER
LogLevel DEBUG
At that point, performing a rescan resulted in an entry in /var/log/messages:
Armed with that entry, I could add the Ciphers entry in sshd_conf, using the options from the Veeam ssh client to the defaults available in this version of sshd:
Ciphers aes128-cbc,blowfish-cbc,3des-cbc,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com
One more rescan, and all was well, making it possible to retry the failed jobs.

Follow Up

There have been responses of both successes and failures from people using this post to get their repository back on line. I'm not sure what's going on, but I'll throw in these additional tips for editing sshd_config:
  1. Each of these entries (KexAlgorithms and Ciphers) are single line entries. You must have the keyword—case sensitive— followed by a single space, followed by the entries without whitespace or breaks.
  2. There's a spot in the default sshd_config that "looks" like the right place to put these entries; that's where I put them. It's a heading labelled "# Ciphers and keying." Just drop them into the space before the Logging section. In the screenshot below, you can see how there's no wrap, no whitespace, etc. This works for me.
  3. Restart the SSH service. You can use the command line (I recommend using telnet during this operation, or you'll loose your SSH connection as the daemon cycles) or the GUI control panel. If using the latter, uncheck SSH, save, check SSH.

27 comments:

  1. Thanks!

    Saved my day and my critical backup

    Best regards from Denmark
    Lars

    ReplyDelete
  2. Great post, but when I add them to the sshd_config ( below all the lines ) my SSHD can't start

    DS1812> synoservicectl --restart sshd
    sshd failed to restart... error unknow [0x0000 (null):0]

    ReplyDelete
    Replies
    1. Found it !

      not below all lines but between de lines :-) THX great post.

      Delete
  3. You saved my day
    Very clever

    ReplyDelete
  4. Great
    Worked for me

    But still issue with veeam
    No offloading to the CPU of the Synology like it used too.

    I made the mistake to update python ... I think there is an issue like that

    ReplyDelete
    Replies
    1. Python isn't used by Veeam, it's all Perl. Not sure which offload you're wanting; maybe you intend it to be a backup proxy in addition to being a repository?

      Delete
  5. This did not work. I cannot start ssh after I edit my sshd_config.
    If it works by you, can you send me your full sshd_config?
    Today I wrote to the Software enginer of Synology between this Problem.
    Synology Support Veeam till 5.0
    https://www.synology.com/de-de/knowledgebase/faq/352

    ReplyDelete
    Replies
    1. Updated post with tips for sshd_config. The only changes from the default is there in those two lines.

      Delete
  6. Worked great, thanks so much.

    Add lines anywhere in config file and make sure to restart your SSH service folks.

    ReplyDelete
  7. Ok,
    now it works... I have to use not the last Lines in the sshd_config

    # obsoletes QuietMode and FascistLogging
    SyslogFacility USER
    LogLevel DEBUG

    # Authentication:

    #LoginGraceTime 2m
    #PermitRootLogin yes
    #StrictModes yes
    #MaxAuthTries 6
    #MaxSessions 10

    #RSAAuthentication yes
    #PubkeyAuthentication yes

    # The default is to check both .ssh/authorized_keys and .ssh/authorized_keys2
    # but this is overridden so installations will only check .ssh/authorized_keys
    #AuthorizedKeysFile .ssh/authorized_keys

    #AuthorizedPrincipalsFile none

    #AuthorizedKeysCommand none
    #AuthorizedKeysCommandUser nobody

    # For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
    #RhostsRSAAuthentication no
    # similar for protocol version 2
    #HostbasedAuthentication no
    # Change to yes if you don't trust ~/.ssh/known_hosts for
    # RhostsRSAAuthentication and HostbasedAuthentication
    #IgnoreUserKnownHosts no
    # Don't read the user's ~/.rhosts and ~/.shosts files
    #IgnoreRhosts yes

    # To disable tunneled clear text passwords, change to no here!
    #PasswordAuthentication yes
    #PermitEmptyPasswords no

    # Change to no to disable s/key passwords
    ChallengeResponseAuthentication no

    # Kerberos options
    #KerberosAuthentication no
    #KerberosOrLocalPasswd yes
    #KerberosTicketCleanup yes
    #KerberosGetAFSToken no

    # GSSAPI options
    #GSSAPIAuthentication no
    #GSSAPICleanupCredentials yes

    # Set this to 'yes' to enable PAM authentication, account processing,
    # and session processing. If this is enabled, PAM authentication will
    # be allowed through the ChallengeResponseAuthentication and
    # PasswordAuthentication. Depending on your PAM configuration,
    # PAM authentication via ChallengeResponseAuthentication may bypass
    # the setting of "PermitRootLogin without-password".
    # If you just want the PAM account and session checks to run without
    # PAM authentication, then enable this but set PasswordAuthentication
    # and ChallengeResponseAuthentication to 'no'.
    UsePAM yes

    #AllowAgentForwarding yes
    AllowTcpForwarding no
    #GatewayPorts no
    #X11Forwarding no
    #X11DisplayOffset 10
    #X11UseLocalhost yes
    #PermitTTY yes
    #PrintMotd yes
    #PrintLastLog yes
    #TCPKeepAlive yes
    #UseLogin no
    UsePrivilegeSeparation sandbox # Default for new installations.
    #PermitUserEnvironment no
    #Compression delayed
    #ClientAliveInterval 0
    #ClientAliveCountMax 3
    UseDNS no
    #PidFile /var/run/sshd.pid
    #MaxStartups 10:30:100
    #PermitTunnel no
    ChrootDirectory none
    #VersionAddendum none

    # no default banner path
    #Banner none

    # override default of no subsystems
    #Subsystem sftp /usr/libexec/sftp-server
    Subsystem sftp internal-sftp -f DAEMON -u 000

    # the following are HPN related configuration options
    # tcp receive buffer polling. disable in non autotuning kernels
    #TcpRcvBufPoll yes

    # disable hpn performance boosts
    #HPNDisabled no

    # buffer size for hpn to non-hpn connections
    #HPNBufferSize 2048


    # allow the use of the none cipher
    #NoneEnabled no
    KexAlgorithms diffie-hellman-group1-sha1,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha1
    Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc

    # Example of overriding settings on a per-user basis
    #Match User anoncvs
    # X11Forwarding no
    # AllowTcpForwarding no
    # PermitTTY no
    # ForceCommand cvs server
    Match User root
    AllowTcpForwarding yes

    ReplyDelete
  8. Putting the KexAlgorithms line in at the ***end*** of the file resulted in the error

    "sshd failed to restart... error unknow [0x0000 (null):0]"

    When I ran /usr/bin/sshd to start the daemon interactively it spat out:
    "/etc/ssh/sshd_config line 151: Directive 'KexAlgorithms' is not allowed within a Match block"

    So, as someone pointed out earlier, move the line up somewhere else in the file and it should work - I put it up near the top.

    ReplyDelete
  9. when i ssh using putting and try to edit the file using the root and admin account, it get permission denied!
    Can you please help.

    ReplyDelete
    Replies
    1. Using root account, you might need to change the permissions (chmod) to make it writeable, but *I* didn't have to. Admin account needs 'sudo' unless permissions are open wider than usual.

      Delete
  10. You sir are awesome! I certainly did not have the energy to dive that deep into this issue this morning. But a couple of lines added to the right file as you instruct, and problem solved. I am however disappointed that a simple DSM update caused such a headache.

    ReplyDelete
  11. Wow, this is what I would call a "job well done"! This fixed it for me. Thank you very much for getting to the bottom of the problem AND providing a quick fix that everybody should be able to perform. First time I've ever been disappointed by Synology for causing such troubles with a simple update and, even worse, denying they caused the problem and not providing us with a solution.

    ReplyDelete
  12. Second time I have had to resource this post. DSM 5.1-1021 broke Veame again. Many, many thanks for the write up!

    ReplyDelete
  13. awesome stuff... thanks so much.. now back to a faster speed with NFS ;)

    ReplyDelete
  14. Thank you very much, I had this error as well ("server does not support diffie-hellman-group1-sha1") after a recent firmware update on our QNAP Nas, and your modifications in sshd_config fixed this problem.

    ReplyDelete
  15. Thanks for psoting this. Had some problems with my QNAP and your post resolved my errors.

    ReplyDelete
  16. Worked for me. Thank you.

    ReplyDelete
  17. Lifesaver! Fixed our QNAP issue with Veeam 8.0.

    ReplyDelete
  18. Thanks a lot. You saved my time :)

    ReplyDelete
  19. I have the synology ds2415 installed DSM 5.1-5022 Update 4 and Veeams 8, getting same error.
    Tried to edit the sshd_config fie but it does not let me.. do I need to login as root .

    have tried commands sudo , su - admin ,

    new to linix, how can I edit the file in the thermal: vi sshd_config
    thanks

    ReplyDelete
  20. Same workaround used for do Backup from Cisco Unified Callmanager. Helped a lot, thank you very much.

    ReplyDelete
  21. The gui has a built in option to do this now. I the screen where you enable/disable SSH, click the "Advanced settings" button, and select "low".

    I'm on DSM 5.2-5592

    ReplyDelete