forkjo/tests/integration
Lunny Xiao aeb383025f
Also sync DB branches on push if necessary (#28361)
Fix #28056 

This PR will check whether the repo has zero branch when pushing a
branch. If that, it means this repository hasn't been synced.

The reason caused that is after user upgrade from v1.20 -> v1.21, he
just push branches without visit the repository user interface. Because
all repositories routers will check whether a branches sync is necessary
but push has not such check.

For every repository, it has two states, synced or not synced. If there
is zero branch for a repository, then it will be assumed as non-sync
state. Otherwise, it's synced state. So if we think it's synced, we just
need to update branch/insert new branch. Otherwise do a full sync. So
that, for every push, there will be almost no extra load added. It's
high performance than yours.

For the implementation, we in fact will try to update the branch first,
if updated success with affect records > 0, then all are done. Because
that means the branch has been in the database. If no record is
affected, that means the branch does not exist in database. So there are
two possibilities. One is this is a new branch, then we just need to
insert the record. Another is the branches haven't been synced, then we
need to sync all the branches into database.
2023-12-09 13:30:56 +00:00
..
migration-test
schemas
actions_trigger_test.go add skip ci functionality (#28075) 2023-11-18 13:37:08 +02:00
admin_config_test.go
admin_user_test.go User details page (#26713) 2023-08-31 11:21:18 +02:00
api_actions_artifact_test.go Artifacts retention and auto clean up (#26131) 2023-09-06 07:41:06 +00:00
api_activitypub_person_test.go More refactoring of db.DefaultContext (#27083) 2023-09-15 06:13:19 +00:00
api_admin_org_test.go
api_admin_test.go show manual cron run's last time (#27544) 2023-10-11 07:28:16 +00:00
api_branch_test.go PATCH branch-protection updates check list even when checks are disabled (#26351) 2023-08-24 05:36:04 +00:00
api_comment_attachment_test.go Fix comment permissions (#28213) 2023-11-25 17:21:21 +00:00
api_comment_test.go Fix comment permissions (#28213) 2023-11-25 17:21:21 +00:00
api_feed_user_test.go
api_fork_test.go
api_gitignore_templates_test.go
api_gpg_keys_test.go
api_helper_for_declarative_test.go
api_httpsig_test.go Expanded minimum RSA Keylength to 3072 (#26604) 2023-08-28 00:53:16 +00:00
api_issue_attachment_test.go
api_issue_config_test.go Fix Bug in Issue Config when only contact links are set (#26521) 2023-09-28 20:16:40 +08:00
api_issue_label_test.go
api_issue_milestone_test.go
api_issue_pin_test.go
api_issue_reaction_test.go Fix comment permissions (#28213) 2023-11-25 17:21:21 +00:00
api_issue_stopwatch_test.go
api_issue_subscription_test.go Even more db.DefaultContext refactor (#27352) 2023-10-03 10:30:41 +00:00
api_issue_test.go Fix review request number and add more tests (#27104) 2023-09-21 13:59:50 +02:00
api_issue_tracked_time_test.go
api_keys_test.go Fix comment permissions (#28213) 2023-11-25 17:21:21 +00:00
api_label_templates_test.go
api_license_templates_test.go
api_nodeinfo_test.go Fix comment permissions (#28213) 2023-11-25 17:21:21 +00:00
api_notification_test.go Even more db.DefaultContext refactor (#27352) 2023-10-03 10:30:41 +00:00
api_oauth2_apps_test.go
api_org_avatar_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
api_org_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
api_packages_alpine_test.go
api_packages_cargo_test.go
api_packages_chef_test.go Fix the wrong HTTP response status code for duplicate packages (#27480) 2023-10-10 15:39:58 +02:00
api_packages_composer_test.go Fix the wrong HTTP response status code for duplicate packages (#27480) 2023-10-10 15:39:58 +02:00
api_packages_conan_test.go Replace assert.Fail with assert.FailNow (#27578) 2023-10-11 11:02:24 +00:00
api_packages_conda_test.go
api_packages_container_test.go Replace assert.Fail with assert.FailNow (#27578) 2023-10-11 11:02:24 +00:00
api_packages_cran_test.go
api_packages_debian_test.go List all Debian package versions in Packages (#27786) 2023-10-29 14:14:47 +00:00
api_packages_generic_test.go
api_packages_goproxy_test.go
api_packages_helm_test.go
api_packages_maven_test.go Fix the wrong HTTP response status code for duplicate packages (#27480) 2023-10-10 15:39:58 +02:00
api_packages_npm_test.go Fix the wrong HTTP response status code for duplicate packages (#27480) 2023-10-10 15:39:58 +02:00
api_packages_nuget_test.go Replace assert.Fail with assert.FailNow (#27578) 2023-10-11 11:02:24 +00:00
api_packages_pub_test.go Fix the wrong HTTP response status code for duplicate packages (#27480) 2023-10-10 15:39:58 +02:00
api_packages_pypi_test.go Fix the wrong HTTP response status code for duplicate packages (#27480) 2023-10-10 15:39:58 +02:00
api_packages_rpm_test.go Add HEAD support for rpm repo files (#28309) 2023-12-05 08:01:02 +00:00
api_packages_rubygems_test.go Fix the wrong HTTP response status code for duplicate packages (#27480) 2023-10-10 15:39:58 +02:00
api_packages_swift_test.go
api_packages_test.go
api_packages_vagrant_test.go
api_private_serv_test.go Final round of db.DefaultContext refactor (#27587) 2023-10-14 08:37:24 +00:00
api_pull_commits_test.go api: GetPullRequestCommits: return file list (#27483) 2023-10-09 11:22:12 +00:00
api_pull_review_test.go More db.DefaultContext refactor (#27265) 2023-09-29 12:12:54 +00:00
api_pull_test.go
api_releases_test.go add Upload URL to release API (#26663) 2023-08-24 10:36:10 +00:00
api_repo_archive_test.go
api_repo_avatar_test.go
api_repo_branch_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
api_repo_collaborator_test.go
api_repo_edit_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
api_repo_file_create_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
api_repo_file_delete_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
api_repo_file_get_test.go
api_repo_file_helpers.go Fix Bug in Issue Config when only contact links are set (#26521) 2023-09-28 20:16:40 +08:00
api_repo_file_update_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
api_repo_files_change_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
api_repo_get_contents_list_test.go Also sync DB branches on push if necessary (#28361) 2023-12-09 13:30:56 +00:00
api_repo_get_contents_test.go Also sync DB branches on push if necessary (#28361) 2023-12-09 13:30:56 +00:00
api_repo_git_blobs_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
api_repo_git_commits_test.go
api_repo_git_hook_test.go
api_repo_git_notes_test.go api: GetPullRequestCommits: return file list (#27483) 2023-10-09 11:22:12 +00:00
api_repo_git_ref_test.go
api_repo_git_tags_test.go
api_repo_git_trees_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
api_repo_hook_test.go
api_repo_languages_test.go
api_repo_lfs_locks_test.go
api_repo_lfs_migrate_test.go
api_repo_lfs_test.go
api_repo_raw_test.go
api_repo_secrets_test.go Refactor secrets modification logic (#26873) 2023-09-05 15:21:02 +00:00
api_repo_tags_test.go
api_repo_teams_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
api_repo_test.go Remove unnecessary parameter (#27671) 2023-10-18 15:03:10 +00:00
api_repo_topic_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
api_settings_test.go
api_team_test.go
api_team_user_test.go
api_token_test.go Fix permissions for Token DELETE endpoint to match GET and POST (#27610) 2023-10-14 08:04:44 +00:00
api_twofa_test.go Unify two factor check (#27915) 2023-11-06 08:22:39 +00:00
api_user_avatar_test.go
api_user_email_test.go
api_user_follow_test.go
api_user_heatmap_test.go
api_user_info_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
api_user_org_perm_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
api_user_orgs_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
api_user_search_test.go Enable system users search via the API (#28013) 2023-11-13 15:31:38 +01:00
api_user_secrets_test.go Add user secrets API integration tests (#27832) 2023-10-31 11:46:09 +08:00
api_user_star_test.go
api_user_watch_test.go
api_wiki_test.go
attachment_test.go
auth_ldap_test.go Use db.Find instead of writing methods for every object (#28084) 2023-11-24 03:49:41 +00:00
avatar.png
benchmarks_test.go
branches_test.go
change_default_branch_test.go
cmd_keys_test.go
compare_test.go
cors_test.go
create_no_session_test.go
csrf_test.go
delete_user_test.go
download_test.go
dump_restore_test.go
editor_test.go
empty_repo_test.go
eventsource_test.go
explore_repos_test.go
git_clone_wiki_test.go
git_helper_for_declarative_test.go
git_smart_http_test.go
git_test.go
goget_test.go
gpg_git_test.go Replace assert.Fail with assert.FailNow (#27578) 2023-10-11 11:02:24 +00:00
html_helper.go
incoming_email_test.go Next round of db.DefaultContext refactor (#27089) 2023-09-16 14:39:12 +00:00
integration_test.go Upgrade to golangci-lint@v1.55.0 (#27756) 2023-10-24 02:54:59 +00:00
issue_test.go Fix comment permissions (#28213) 2023-11-25 17:21:21 +00:00
lfs_getobject_test.go Remove GetByBean method because sometimes it's danger when query condition parameter is zero and also introduce new generic methods (#28220) 2023-12-07 15:27:36 +08:00
lfs_local_endpoint_test.go
lfs_view_test.go
links_test.go Test more templates for if they contain an error (#27367) 2023-10-02 07:56:55 +00:00
markup_external_test.go
migrate_test.go
mirror_pull_test.go Fix comment permissions (#28213) 2023-11-25 17:21:21 +00:00
mirror_push_test.go move repository deletion to service layer (#26948) 2023-09-08 04:51:15 +00:00
nonascii_branches_test.go
oauth_test.go
org_count_test.go Use db.Find instead of writing methods for every object (#28084) 2023-11-24 03:49:41 +00:00
org_project_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
org_team_invite_test.go Update team invitation email link (#26550) 2023-08-31 16:26:13 +00:00
org_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
private-testing.key
privateactivity_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
pull_compare_test.go
pull_create_test.go
pull_diff_test.go
pull_merge_test.go Final round of db.DefaultContext refactor (#27587) 2023-10-14 08:37:24 +00:00
pull_review_test.go
pull_status_test.go Restore warning commit status (#27504) 2023-10-08 22:16:06 +00:00
pull_update_test.go Move createrepository from module to service layer (#26927) 2023-09-06 12:08:51 +00:00
README.md Require MySQL 8.0, PostgreSQL 12, MSSQL 2012 (#27337) 2023-10-03 13:27:57 -04:00
README_ZH.md Require MySQL 8.0, PostgreSQL 12, MSSQL 2012 (#27337) 2023-10-03 13:27:57 -04:00
release_test.go Allow get release download files and lfs files with oauth2 token format (#26430) 2023-10-01 10:41:52 +00:00
rename_branch_test.go
repo_activity_test.go
repo_branch_test.go
repo_commits_search_test.go
repo_commits_test.go Restore warning commit status (#27504) 2023-10-08 22:16:06 +00:00
repo_fork_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
repo_generate_test.go
repo_migrate_test.go
repo_search_test.go
repo_tag_test.go
repo_test.go Add support for HEAD ref in /src/branch and /src/commit routes (#27384) 2023-10-03 15:37:06 +08:00
repo_topic_test.go
repo_watch_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
repofiles_change_test.go Move web/api context related testing function into a separate package (#26859) 2023-09-01 11:26:07 +00:00
setting_test.go
signin_test.go Enhanced auth token / remember me (#27606) 2023-10-14 00:56:41 +00:00
signout_test.go
signup_test.go
ssh_key_test.go
timetracking_test.go Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052) 2023-09-14 02:59:53 +00:00
user_avatar_test.go
user_test.go
version_test.go
view_test.go
webfinger_test.go
xss_test.go

Integration tests

Integration tests can be run with make commands for the appropriate backends, namely:

make test-sqlite
make test-pgsql
make test-mysql
make test-mssql

Make sure to perform a clean build before running tests:

make clean build

Run tests via local act_runner

Run all jobs

act_runner exec -W ./.github/workflows/pull-db-tests.yml --event=pull_request --default-actions-url="https://github.com" -i catthehacker/ubuntu:runner-latest

Warning: This file defines many jobs, so it will be resource-intensive and therefor not recommended.

Run single job

act_runner exec -W ./.github/workflows/pull-db-tests.yml --event=pull_request --default-actions-url="https://github.com" -i catthehacker/ubuntu:runner-latest -j <job_name>

You can list all job names via:

act_runner exec -W ./.github/workflows/pull-db-tests.yml --event=pull_request --default-actions-url="https://github.com" -i catthehacker/ubuntu:runner-latest -l

Run sqlite integration tests

Start tests

make test-sqlite

Run MySQL integration tests

Setup a MySQL database inside docker

docker run -e "MYSQL_DATABASE=test" -e "MYSQL_ALLOW_EMPTY_PASSWORD=yes" -p 3306:3306 --rm --name mysql mysql:latest #(just ctrl-c to stop db and clean the container)
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" --rm --name elasticsearch elasticsearch:7.6.0 #(in a second terminal, just ctrl-c to stop db and clean the container)

Start tests based on the database container

TEST_MYSQL_HOST=localhost:3306 TEST_MYSQL_DBNAME=test TEST_MYSQL_USERNAME=root TEST_MYSQL_PASSWORD='' make test-mysql

Run pgsql integration tests

Setup a pgsql database inside docker

docker run -e "POSTGRES_DB=test" -p 5432:5432 --rm --name pgsql postgres:latest #(just ctrl-c to stop db and clean the container)

Start tests based on the database container

TEST_PGSQL_HOST=localhost:5432 TEST_PGSQL_DBNAME=test TEST_PGSQL_USERNAME=postgres TEST_PGSQL_PASSWORD=postgres make test-pgsql

Run mssql integration tests

Setup a mssql database inside docker

docker run -e "ACCEPT_EULA=Y" -e "MSSQL_PID=Standard" -e "SA_PASSWORD=MwantsaSecurePassword1" -p 1433:1433 --rm --name mssql microsoft/mssql-server-linux:latest #(just ctrl-c to stop db and clean the container)

Start tests based on the database container

TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=gitea_test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-mssql

Running individual tests

Example command to run GPG test:

For SQLite:

make test-sqlite#GPG

For other databases(replace mssql to mysql, or pgsql):

TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-mssql#GPG

Setting timeouts for declaring long-tests and long-flushes

We appreciate that some testing machines may not be very powerful and the default timeouts for declaring a slow test or a slow clean-up flush may not be appropriate.

You can either:

  • Within the test ini file set the following section:
[integration-tests]
SLOW_TEST = 10s ; 10s is the default value
SLOW_FLUSH = 5S ; 5s is the default value
  • Set the following environment variables:
GITEA_SLOW_TEST_TIME="10s" GITEA_SLOW_FLUSH_TIME="5s" make test-sqlite