Difference between revisions of "Developing Services"

From Dark Peak
Jump to: navigation, search
(From the 2017 AGM, added a section on a new approach to getting more people involved in developing and maintaining services)
 
(5 intermediate revisions by one other user not shown)
Line 1: Line 1:
Users, and possibly other services, need to be able to understand what they can expect from a service.
+
== Service development teams ==
In particular, how well it can be relied upon.
 
To do that we define maturity levels of services ([[Alpha]], [[Beta]], [[Live]]) and define types of service based upon who/what consumes  them ([[User Service|User]], [[Co-op Service|Co-Op]], [[Infrastructure]]).
 
 
 
These definitions and processes attempt to keep things:
 
;Minimal
 
:Avoid over specifying
 
:Limit special cases
 
;Local
 
:where possible, grant autonomy to the people responsible for the service.
 
;Clarity of requirements to maintainers
 
:What is required at each level
 
:What is provided by the Co-op to help achieve these requirements.
 
;Clarity of service to those outside the set of maintainers
 
:What’s used to create the service
 
:What’s provided to the users of the service
 
:How the service aligns with co-op principles
 
;End-user focused
 
:What the user can expect of the service expect in terms of Service Quality and long term availability
 
:Build only what’s needed, if indirectly, by the end user
 
 
== Definitions ==
 
 
 
Types
 
;Infrastructure
 
:A service other services rely on
 
;Co-Op
 
:A service the co-op relies on to function, directly or indirectly.
 
;User
 
:A service that only exists for our members to use
 
 
 
{|class="wikitable"
 
| Level
 
| State
 
| 9’s of uptime (acceptable downtime/year)
 
| Data retention/backups
 
|-
 
| Alpha
 
| Buildable
 
| 0
 
| None
 
|-
 
| Beta
 
| Usable
 
| 1 (36 days)
 
| Some, clearly defined by the service
 
|-
 
| Live
 
| Reliable
 
| 2 (3.6 days)
 
| Full backup and restore
 
|}
 
 
 
== Level Requirements ==
 
 
 
=== Alpha ===
 
Demonstration of interest:
 
: Infrastructure    At least one existing or alpha service that would use it, and board agreement to provide.
 
: Co-Op        Agreement that this is something the co-op needs
 
: User        Sufficient users say they would find the service personally valuable
 
 
 
Have a Loomio thread, wiki page and version control plan.
 
 
 
Basic hosting plan in wiki including at least:
 
* what services you expect to use
 
* How this service respects the co-op principles
 
* any expected oddities (high resource use, new infrastructure needed, etc)
 
 
 
=== Beta ===
 
All requirements for Alpha met plus:
 
 
 
Running usable instance.
 
 
 
Minimum required maintainers:
 
: Infrastructure    at least 1 Board Member + 2 other members.
 
: Co-Op        at least 1 Board Member + 2 other members.
 
: User        at least 2 members
 
 
 
No reliance on any alpha infrastructure
 
 
 
Meets all its infrastructures requirements.
 
 
 
All service and configuration details stored in git with appropriate levels of access
 
 
 
Documentation in the wiki:
 
* Service design and decisions
 
** What external resources are used e.g. what software you installed
 
** Infrastructure or alternatives used and why
 
** Major bits of configuration and why
 
* how to use the service
 
* Data backup, retention and restoration policies
 
* if infrastructure what you provide to client services and how to get them
 
  
 +
Two people should be involved in developing a new [[User Service]]: someone who wants the service to exist and is keen to learn how to make it work; and another person with knowledge of how things work at present will be found to work with you on developing the alpha. They should arrange to get together in person to hack out the alpha version - our hack days are ideal for this purpose.
  
=== Live ===
+
The benefits of this approach are that less experienced members can learn from those with more knowledge and skills; and we start to reduce the burden on our more knowledgeable members by bringing others up to speed on how to help them manage services.
All requirements for Beta met plus:
 
  
No reliance on any beta infrastructure.
+
== Service production levels ==
  
Automated backup.
+
* Decide if your new service will be a [[User Service]], [[Co-op Service]] or [[Infrastructure]].
 +
* Work to reach [[Alpha]] level, this should take a few hours. This is about validating the idea and thinking through what you'll do and what you need to do it.
 +
* Ask the co-op to verify that you've achieved this
 +
* Work to reach [[Beta]] level. This should be the bulk of the deployment work, and involves finding some co-maintainers
 +
* Ask the co-op to verify that you've achieved this
 +
* Work to reach [[Live]]. This is mostly about simplifying future maintenance work and ensuring it will get done.
 +
* Ask the co-op to verify that you've achieved this
  
A maintenance / ops guide on the wiki with instructions that all the maintainers feels happy following for:
+
These levels exist so that users of your service are able to understand what they can expect from a service.
* service restart
+
There is no requirement that all services reach live, if all your users are happy with something less reliable you can remain at Beta indefinitely.
* upgrade
 
* restore from a backup
 
* Remove
 
* Migrate an existing instance
 
* deploy new instances
 
* fixing expected problems and events
 
* Maintenance and rollback plans/processes
 
  
Any info (including keys/passwords) needed to access and admin the service must be provided to the board to be stored using the key management tools.
+
Running Services are expected to provide an annual at the AGM that indicates whether the service continues to meet the requirements for the maturity level it is operating at.
  
Infrastructure must define how any functionality provided to another service can be cleanly removed if needed, what default levels of resources it will provide at each level and how that will be enforced/monitored. This can include requiring information from the client. Anything too secure to be provided up front e.g. passwords can instead be escrowed with the board/membership.
+
Services which start to fail to meet their requirements have until the next general meeting to correct the issues.
 
+
If they continue to fail to meet the requirements means the service descends one maturity level.
Explanation of the service’s security and how you manage access to any particularly sensitive data.
+
Such a degradation may place other services also under scrutiny; for example infrastructure being degraded would also affect any services consuming that infrastructure.  
 
 
== Process ==
 
Ascending the maturity levels is validated with a co-op vote checking that the service has met the requirements for the new maturity level.
 
Whilst this is not a subjective vote on the quality of how requirements are met, there is an expectation that discussion will occur between members prior to a vote occurring.
 
This should ensure that the quality of how requirements are met remains acceptable.
 
 
 
Services are expected to provide an annual update (via the AGM) that indicates whether the service continues to meet the requirements for the maturity level it is operating at.
 
Services which start to fail to meet their requirements are given a reasonable period of time to correct the deficiencies (until the next general meeting); continue to fail to meet the requirements means the service descends one maturity level.
 
\Such a degradation may place other services also under scrutiny; for example infrastructure being degraded would also affect any services consuming that infrastructure.  
 
 
In addition, live service maintainers have the responsibility to notify the co-op of a change which would mean that the service no longer meets its live requirements.
 
In addition, live service maintainers have the responsibility to notify the co-op of a change which would mean that the service no longer meets its live requirements.
 
The board should act on these notifications by following the same process as above (ie - entering the “reasonable” grace period of time to correct the deficiencies).
 
The board should act on these notifications by following the same process as above (ie - entering the “reasonable” grace period of time to correct the deficiencies).
Line 129: Line 28:
 
Level requirements will only change at the the AGM or an EGM.
 
Level requirements will only change at the the AGM or an EGM.
 
Services then have until the next AGM to meet these requirements. If that is failed then the above process is followed.
 
Services then have until the next AGM to meet these requirements. If that is failed then the above process is followed.
 +
 +
== Meta ==
 +
 +
These definitions and processes attempt to keep things:
 +
; Minimal
 +
: Avoid over specifying, set broad targets that a service can meet many ways
 +
: Limit special cases, these should be specific agreements at the service level rather than added to the requirements
 +
; Local
 +
: Where possible, grant autonomy to the people responsible for the service.
 +
; Clarity of requirements to maintainers
 +
: What is required at each level
 +
: What is provided by the Co-op to help achieve these requirements.
 +
; Clarity of service to those outside the set of maintainers
 +
: What’s used to create the service
 +
: What’s provided to the users of the service
 +
: How the service aligns with co-op principles
 +
; End-user focused
 +
: what can the user of the service expect in terms of Service Quality and  how likely the service is to still be available in the future.
 +
: Build only what’s needed, if indirectly, by the end user
 +
 +
Where we fail to meet these goals the relevant requirements should be updated.

Latest revision as of 14:41, 11 November 2017

Service development teams

Two people should be involved in developing a new User Service: someone who wants the service to exist and is keen to learn how to make it work; and another person with knowledge of how things work at present will be found to work with you on developing the alpha. They should arrange to get together in person to hack out the alpha version - our hack days are ideal for this purpose.

The benefits of this approach are that less experienced members can learn from those with more knowledge and skills; and we start to reduce the burden on our more knowledgeable members by bringing others up to speed on how to help them manage services.

Service production levels

  • Decide if your new service will be a User Service, Co-op Service or Infrastructure.
  • Work to reach Alpha level, this should take a few hours. This is about validating the idea and thinking through what you'll do and what you need to do it.
  • Ask the co-op to verify that you've achieved this
  • Work to reach Beta level. This should be the bulk of the deployment work, and involves finding some co-maintainers
  • Ask the co-op to verify that you've achieved this
  • Work to reach Live. This is mostly about simplifying future maintenance work and ensuring it will get done.
  • Ask the co-op to verify that you've achieved this

These levels exist so that users of your service are able to understand what they can expect from a service. There is no requirement that all services reach live, if all your users are happy with something less reliable you can remain at Beta indefinitely.

Running Services are expected to provide an annual at the AGM that indicates whether the service continues to meet the requirements for the maturity level it is operating at.

Services which start to fail to meet their requirements have until the next general meeting to correct the issues. If they continue to fail to meet the requirements means the service descends one maturity level. Such a degradation may place other services also under scrutiny; for example infrastructure being degraded would also affect any services consuming that infrastructure. In addition, live service maintainers have the responsibility to notify the co-op of a change which would mean that the service no longer meets its live requirements. The board should act on these notifications by following the same process as above (ie - entering the “reasonable” grace period of time to correct the deficiencies).

Level requirements will only change at the the AGM or an EGM. Services then have until the next AGM to meet these requirements. If that is failed then the above process is followed.

Meta

These definitions and processes attempt to keep things:

Minimal
Avoid over specifying, set broad targets that a service can meet many ways
Limit special cases, these should be specific agreements at the service level rather than added to the requirements
Local
Where possible, grant autonomy to the people responsible for the service.
Clarity of requirements to maintainers
What is required at each level
What is provided by the Co-op to help achieve these requirements.
Clarity of service to those outside the set of maintainers
What’s used to create the service
What’s provided to the users of the service
How the service aligns with co-op principles
End-user focused
what can the user of the service expect in terms of Service Quality and how likely the service is to still be available in the future.
Build only what’s needed, if indirectly, by the end user

Where we fail to meet these goals the relevant requirements should be updated.