19d9d4481SDarren TuckerOverview.
29d9d4481SDarren Tucker
39d9d4481SDarren Tucker$ ./configure && make tests
49d9d4481SDarren Tucker
59d9d4481SDarren TuckerYou'll see some progress info. A failure will cause either the make to
69d9d4481SDarren Tuckerabort or the driver script to report a "FATAL" failure.
79d9d4481SDarren Tucker
89d9d4481SDarren TuckerThe test consists of 2 parts. The first is the file-based tests which is
99d9d4481SDarren Tuckerdriven by the Makefile, and the second is a set of network or proxycommand
109d9d4481SDarren Tuckerbased tests, which are driven by a driver script (test-exec.sh) which is
119d9d4481SDarren Tuckercalled multiple times by the Makefile.
129d9d4481SDarren Tucker
139d9d4481SDarren TuckerFailures in the first part will cause the Makefile to return an error.
149d9d4481SDarren TuckerFailures in the second part will print a "FATAL" message for the failed
159d9d4481SDarren Tuckertest and continue.
169d9d4481SDarren Tucker
179d9d4481SDarren TuckerOpenBSD has a system-wide regression test suite. OpenSSH Portable's test
189d9d4481SDarren Tuckersuite is based on OpenBSD's with modifications.
199d9d4481SDarren Tucker
209d9d4481SDarren Tucker
219d9d4481SDarren TuckerEnvironment variables.
229d9d4481SDarren Tucker
239617816dSDamien MillerSKIP_UNIT: Skip unit tests.
249617816dSDamien MillerSUDO: path to sudo/doas command, if desired. Note that some systems
259617816dSDamien Miller	(notably systems using PAM) require sudo to execute some tests.
269617816dSDamien MillerLTESTS: Whitespace separated list of tests (filenames without the .sh
279617816dSDamien Miller	extension) to run.
28*0a403bfdSDarren TuckerSKIP_LTESTS: Whitespace separated list of tests to skip.
299d9d4481SDarren TuckerOBJ: used by test scripts to access build dir.
307a06f62bSDarren TuckerTEST_SHELL: shell used for running the test scripts.
319617816dSDamien MillerTEST_SSH_FAIL_FATAL: set to "yes" to make any failure abort the test
329617816dSDamien Miller	currently in progress.
33f9eb2b01SDarren TuckerTEST_SSH_PORT: TCP port to be used for the listening tests.
349617816dSDamien MillerTEST_SSH_QUIET: set to "yes" to suppress non-fatal output.
35d9b9b436SDarren TuckerTEST_SSH_SSHD_CONFOPTS: Configuration directives to be added to sshd_config
36f9eb2b01SDarren Tucker	before running each test.
379617816dSDamien MillerTEST_SSH_SSH_CONFOPTS: Configuration directives to be added to
389617816dSDamien Miller	ssh_config before running each test.
399617816dSDamien MillerTEST_SSH_TRACE: set to "yes" for verbose output from tests
409617816dSDamien MillerTEST_SSH_x: path to "ssh" command under test, where x is one of
429617816dSDamien Miller	SFTPSERVER
439617816dSDamien MillerUSE_VALGRIND: Run the tests under valgrind memory checker.
449d9d4481SDarren Tucker
459d9d4481SDarren Tucker
469d9d4481SDarren TuckerIndividual tests.
479d9d4481SDarren Tucker
4813fbe577SDarren TuckerYou can run an individual test from the top-level Makefile, eg:
4913fbe577SDarren Tucker$ make tests LTESTS=agent-timeout
5013fbe577SDarren Tucker
5113fbe577SDarren TuckerIf you need to manipulate the environment more you can invoke test-exec.sh
5213fbe577SDarren Tuckerdirectly if you set up the path to find the binaries under test and the
5313fbe577SDarren Tuckertest scripts themselves, for example:
549d9d4481SDarren Tucker
559d9d4481SDarren Tucker$ cd regress
567a06f62bSDarren Tucker$ PATH=`pwd`/..:$PATH:. TEST_SHELL=/bin/sh sh test-exec.sh `pwd` \
577a06f62bSDarren Tucker    agent-timeout.sh
589d9d4481SDarren Tuckerok agent timeout test
599d9d4481SDarren Tucker
609d9d4481SDarren Tucker
619d9d4481SDarren TuckerFiles.
629d9d4481SDarren Tucker
639d9d4481SDarren Tuckertest-exec.sh: the main test driver. Sets environment, creates config files
649d9d4481SDarren Tuckerand keys and runs the specified test.
659d9d4481SDarren Tucker
669d9d4481SDarren TuckerAt the time of writing, the individual tests are:
679d9d4481SDarren Tuckerconnect.sh:		simple connect
689d9d4481SDarren Tuckerproxy-connect.sh:	proxy connect
6958ceacdcSJitendra Sharmaconnect-privsep.sh:	proxy connect with privsep
7058ceacdcSJitendra Sharmaconnect-uri.sh:		uri connect
7158ceacdcSJitendra Sharmaproto-version.sh:	sshd version with different protocol combinations
7258ceacdcSJitendra Sharmaproto-mismatch.sh:	protocol version mismatch
7358ceacdcSJitendra Sharmaexit-status.sh:		remote exit status
7458ceacdcSJitendra Sharmaenvpass.sh:		environment passing
7558ceacdcSJitendra Sharmatransfer.sh:		transfer data
7658ceacdcSJitendra Sharmabanner.sh:		banner
7758ceacdcSJitendra Sharmarekey.sh:		rekey
7858ceacdcSJitendra Sharmastderr-data.sh:		stderr data transfer
7958ceacdcSJitendra Sharmastderr-after-eof.sh:	stderr data after eof
8058ceacdcSJitendra Sharmabroken-pipe.sh:		broken pipe test
8158ceacdcSJitendra Sharmatry-ciphers.sh:		try ciphers
8258ceacdcSJitendra Sharmayes-head.sh:		yes pipe head
8358ceacdcSJitendra Sharmalogin-timeout.sh:	connect after login grace timeout
8458ceacdcSJitendra Sharmaagent.sh:		simple connect via agent
8558ceacdcSJitendra Sharmaagent-getpeereid.sh:	disallow agent attach from other uid
8658ceacdcSJitendra Sharmaagent-timeout.sh:	agent timeout test
8758ceacdcSJitendra Sharmaagent-ptrace.sh:	disallow agent ptrace attach
8858ceacdcSJitendra Sharmakeyscan.sh:		keyscan
8958ceacdcSJitendra Sharmakeygen-change.sh:	change passphrase for key
9058ceacdcSJitendra Sharmakeygen-convert.sh:	convert keys
9158ceacdcSJitendra Sharmakeygen-moduli.sh:	keygen moduli
9258ceacdcSJitendra Sharmakey-options.sh:		key options
9358ceacdcSJitendra Sharmascp.sh:			scp
9458ceacdcSJitendra Sharmascp-uri.sh:		scp-uri
959d9d4481SDarren Tuckersftp.sh:		basic sftp put/get
9658ceacdcSJitendra Sharmasftp-chroot.sh:		sftp in chroot
9758ceacdcSJitendra Sharmasftp-cmds.sh:		sftp command
9858ceacdcSJitendra Sharmasftp-badcmds.sh:	sftp invalid commands
9958ceacdcSJitendra Sharmasftp-batch.sh:		sftp batchfile
10058ceacdcSJitendra Sharmasftp-glob.sh:		sftp glob
10158ceacdcSJitendra Sharmasftp-perm.sh:		sftp permissions
10258ceacdcSJitendra Sharmasftp-uri.sh:		sftp-uri
1039d9d4481SDarren Tuckerssh-com-client.sh:	connect with ssh.com client
1049d9d4481SDarren Tuckerssh-com-keygen.sh:	ssh.com key import
1059d9d4481SDarren Tuckerssh-com-sftp.sh:	basic sftp put/get with ssh.com server
1069d9d4481SDarren Tuckerssh-com.sh:		connect to ssh.com server
10758ceacdcSJitendra Sharmareconfigure.sh:		simple connect after reconfigure
10858ceacdcSJitendra Sharmadynamic-forward.sh:	dynamic forwarding
10958ceacdcSJitendra Sharmaforwarding.sh:		local and remote forwarding
11058ceacdcSJitendra Sharmamultiplex.sh:		connection multiplexing
11158ceacdcSJitendra Sharmareexec.sh:		reexec tests
11258ceacdcSJitendra Sharmabrokenkeys.sh:		broken keys
11358ceacdcSJitendra Sharmasshcfgparse.sh:		ssh config parse
11458ceacdcSJitendra Sharmacfgparse.sh:		sshd config parse
11558ceacdcSJitendra Sharmacfgmatch.sh:		sshd_config match
11658ceacdcSJitendra Sharmacfgmatchlisten.sh:	sshd_config matchlisten
11758ceacdcSJitendra Sharmaaddrmatch.sh:		address match
11858ceacdcSJitendra Sharmalocalcommand.sh:	localcommand
11958ceacdcSJitendra Sharmaforcecommand.sh:	forced command
12058ceacdcSJitendra Sharmaportnum.sh:		port number parsing
12158ceacdcSJitendra Sharmakeytype.sh:		login with different key types
12258ceacdcSJitendra Sharmakextype.sh:		login with different key exchange algorithms
12358ceacdcSJitendra Sharmacert-hostkey.sh		certified host keys
12458ceacdcSJitendra Sharmacert-userkey.sh:	certified user keys
12558ceacdcSJitendra Sharmahost-expand.sh:		expand %h and %n
12658ceacdcSJitendra Sharmakeys-command.sh:	authorized keys from command
12758ceacdcSJitendra Sharmaforward-control.sh:	sshd control of local and remote forwarding
12858ceacdcSJitendra Sharmaintegrity.sh:		integrity
12958ceacdcSJitendra Sharmakrl.sh:			key revocation lists
13058ceacdcSJitendra Sharmamultipubkey.sh:		multiple pubkey
13158ceacdcSJitendra Sharmalimit-keytype.sh:	restrict pubkey type
13258ceacdcSJitendra Sharmahostkey-agent.sh:	hostkey agent
13358ceacdcSJitendra Sharmakeygen-knownhosts.sh:	ssh-keygen known_hosts
13458ceacdcSJitendra Sharmahostkey-rotate.sh:	hostkey rotate
13558ceacdcSJitendra Sharmaprincipals-command.sh:	authorized principals command
13658ceacdcSJitendra Sharmacert-file.sh:		ssh with certificates
13758ceacdcSJitendra Sharmacfginclude.sh:		config include
13858ceacdcSJitendra Sharmaallow-deny-users.sh:	AllowUsers/DenyUsers
13958ceacdcSJitendra Sharmaauthinfo.sh:		authinfo
1409d9d4481SDarren Tucker
1419d9d4481SDarren Tucker
1429d9d4481SDarren TuckerProblems?
1439d9d4481SDarren Tucker
1449d9d4481SDarren TuckerRun the failing test with shell tracing (-x) turned on:
1459d9d4481SDarren Tucker$ PATH=`pwd`/..:$PATH:. sh -x test-exec.sh `pwd` agent-timeout.sh
1469d9d4481SDarren Tucker
1479d9d4481SDarren TuckerFailed tests can be difficult to diagnose. Suggestions:
1489d9d4481SDarren Tucker- run the individual test via ./test-exec.sh `pwd` [testname]
1499d9d4481SDarren Tucker- set LogLevel to VERBOSE in test-exec.sh and enable syslogging of
1509d9d4481SDarren Tucker  auth.debug (eg to /var/log/authlog).
1519d9d4481SDarren Tucker
1529d9d4481SDarren Tucker
1539d9d4481SDarren TuckerKnown Issues.
1549d9d4481SDarren Tucker
1551f85dc70SDarren Tucker- Similarly, if you do not have "scp" in your system's $PATH then the
1561f85dc70SDarren Tucker  multiplex scp tests will fail (since the system's shell startup scripts
1571f85dc70SDarren Tucker  will determine where the shell started by sshd will look for scp).
1581f85dc70SDarren Tucker
159bad5f2d3SDarren Tucker- Recent GNU coreutils deprecate "head -[n]": this will cause the yes-head
160bad5f2d3SDarren Tucker  test to fail.  The old behaviour can be restored by setting (and
161bad5f2d3SDarren Tucker  exporting) _POSIX2_VERSION=199209 before running the tests.