About Beta testing
Beta testing is where you'll conduct user acceptance testing (UAT) on your service. This version of your service is as close to final development as possible, but it still needs 1 important input - user feedback!
Testing your service with real people means you'll be able to uncover bugs, barriers, and identify solutions and areas for improvement. You'll share this feedback with your developer and iterate the service to include these improvements.
You will test your service in restricted and unrestricted Private Beta before it moves to Public Beta testing and live.
Organize and run restricted Private Beta testing
No one can access a service or website in restricted private beta without your permission. This phase of testing allows you to start small and get quick feedback from users before rolling your service or website out to a wider audience.
What to do in restricted Private Beta
- Have your developer move your service or website to a test server. Avoid development or production servers and servers solely controlled by external vendors. If your service or website includes different components, for example a back-end system or API, move those components to test environments. Do not mix test and production data and connections.
- Email [email protected] to protect the service or website with a network filter or a username and password. With a username and password, you should have a version for the project team and another that you can share with more people.
- When applicable, provide eServices with an email address from which service messages to users can be sent and received. Once they set this up, test to ensure the email address can send and receive messages.
- Ensure the feedback form or email link on your service or website is working. Have members of your project team submit feedback and confirm that you receive their submissions.
- Email your project team and others involved in testing that the service is ready for their review. Include information outlined in the guidelines for emailing test participants section.
- Monitor user feedback and follow the steps in the triage testing feedback and iterate section.
Scope and execute unrestricted Private Beta testing
The next step is to open your service or website up to more people in unrestricted private beta. For example, you can invite a larger group of clients or limit access to a handful of departments.
What to do in unrestricted Private Beta
- The vendor will move your service or website to a production server. Related components should also move to production. The goal is to replicate the final user experience as much as possible.
- Make sure you're set up to measure and report on the service or website's performance. Your eServices delivery manager will coordinate getting you the required analytics tracking code. Share the code with your vendor and they'll add it to your service or website. Once this is done, let your eSservices delivery manager know so they cane confirm data is being collected.
- The service or website owner will coordinate test transactions through the service and ensure submissions safely make it through. Records should appear in the back-end system and get displayed in any merchant account reports. Verify the Department of Finance's reports show the payment transaction in their General Ledger account.
- Verify with the service owner that they can view related audit logs and will have a plan in place to review these once a month.
- Email [email protected] to remove the network filter or username and password protection they put in place during restricted private beta testing.
- After the access restrictions are removed, ask your project team to confirm they can access the service or website by opening a web browser tab in private or incognito mode. They shouldn’t have to enter a username and password to access the service.
- Decide how many people you want to invite to test the service. The number of people will help you determine how you contact them to participate.
- Monitor user feedback and follow the steps in the triage testing feedback and iterate section.
Guidance for emailing user testing participants
Government-wide, internal testing
Use an internal, government-wide Global Note in instances where the service or website is of great importance and needs to be tested with a large pool of internal users.
Global Notes are organized by the Public Service Commission. You must get your director and department communications unit's approval to send a Global Note.
Testing with staff from a few departments
Work with your department communications unit to identify which department email groups to use. They will help you draft your email message and follow department approvals.
Information to include in your email
Global Notes and all email invitations should include the following information:
- a URL to the service;
- a username and password they can use to access the service or website for restricted beta testing;
- a reminder that this is a private, staff-only beta. Ask them to not share the URL with anyone else without your permission;
- details on how long access will be open. Depending on the service, access should last between 1 and 2 weeks. A limited period encourages people to get involved rather than wait; and
- instructions for how they can submit feedback. Mention that their feedback will be reviewed, organized and prioritized.
Triage testing feedback and iterate
Triage your user feedback at the end of restricted and unrestricted private beta testing.
- Review the submissions and document them in the backlog.
- Label each item as a bug, improvement, or new feature.
- Prioritize items by their value and complexity.
Iterate your service or website at the end of restricted and unrestricted private beta testing.
- Make critical improvements to the private beta.
- Inform users that if their feedback was received but not acted upon, what will happen to their input.
- Figure out how to notify users about the new version, including details on what was changed based upon feedback.
Go back to: Develop a Beta
Next step: Public Beta testing and live