OnCallReady

Lesson 31.26 · AWS III: CloudWatch, CloudTrail, Cost & Incidents · 17 min read

Organizations and service control policies

In plain words

A company does not run one AWS account but dozens, like a hotel with many rooms. AWS Organizations is the hotel's front desk: it groups the rooms into floors (organizational units) and sets house rules that apply to everyone on a floor.

Those house rules are service control policies. They never give anyone a key; they only say what nobody on that floor may ever do - not even the room's own manager, who otherwise can do everything inside the room.

Organizations and service control policies

Real companies do not run one AWS account; they run dozens - one per team and environment - grouped in AWS Organizations. The account is the hard boundary (its own IAM, quotas, bill and blast radius), and the organization puts guardrails over all of them: service control policies (SCPs) that no admin inside an account can get around. AWS I met an SCP from the receiving end (an AccessDenied "with an explicit deny in a service control policy"); this lesson is the other side: reading the tree, reading what applies to an account, and changing a guardrail without locking everyone out.

Need to know: an organization has one management account (it pays, and SCPs never apply to it), a root, OUs (folders: Security, Workloads/Prod ...) and member accounts. An SCP never grants anything; it caps what every principal in the accounts below it may do - the root user included. For a call to be allowed, every level between the root and the account needs an SCP that allows it (AWS attaches FullAWSAccess everywhere), and no SCP on the path may deny it. Deny-list SCPs with NotAction must exempt the global services (IAM, Organizations, Cost Explorer, Budgets, Route 53, Support ...) when they restrict Regions.

The tree

Any member account can read its organization's basics. This account is also the organization's delegated administrator for policies - the management account wrote a resource-based delegation policy that lets it read the tree and manage SCPs and tag policies:

$ aws organizations describe-organization --query 'Organization.[Id,MasterAccountId,FeatureSet]' --output text
o-a1b2c3d4e5	444455556666	ALL
$ aws organizations describe-resource-policy --query 'ResourcePolicy.Content' --output text | jq -c '.Statement[0] | {who: .Principal, actions: (.Action | length), types: .Condition}'
{"who":{"AWS":"arn:aws:iam::111122223333:root"},"actions":24,"types":{"StringLikeIfExists":{"organizations:PolicyType":["SERVICE_CONTROL_POLICY","TAG_POLICY"]}}}
$ aws organizations list-organizational-units-for-parent --parent-id r-f6g7 --query 'OrganizationalUnits[].[Id,Name]' --output text
ou-f6g7-4h2kq9sz	Security
ou-f6g7-8m3tw1ab	Platform
ou-f6g7-2c9vx5df	Workloads
ou-f6g7-9b5ue2mn	Sandbox
$ aws organizations list-accounts --query 'Accounts[].[Id,Name,State]' --output text
444455556666	oncall-lab-management	ACTIVE
210987654321	log-archive	ACTIVE
555566667777	audit	ACTIVE
111122223333	oncall-lab	ACTIVE
777788889999	shop-prod	ACTIVE
222233334444	shop-staging	ACTIVE
333344445555	sandbox-1	ACTIVE

Without a delegation, a member account gets AccessDeniedException: You don't have permissions to access this resource. for everything but describe-organization; the management account can do everything - which is why it should run nothing else, have very few people with access, and be the one place you do not deploy workloads. Where is this account in the tree?

$ aws organizations list-parents --child-id 111122223333
{
    "Parents": [
        {
            "Id": "ou-f6g7-8m3tw1ab",
            "Type": "ORGANIZATIONAL_UNIT"
        }
    ]
}
$ aws organizations list-organizational-units-for-parent --parent-id ou-f6g7-2c9vx5df --query 'OrganizationalUnits[].Name' --output text
Prod	Staging

What applies to an account

The SCPs that limit an account are those attached to the root, to every OU on its path, and to the account itself. list-policies-for-target lists what is attached directly to one target, so walk the path:

$ for t in r-f6g7 ou-f6g7-8m3tw1ab 111122223333; do echo "== $t"; aws organizations list-policies-for-target --target-id $t --filter SERVICE_CONTROL_POLICY --query 'Policies[].[Id,Name,AwsManaged]' --output text; done
== r-f6g7
p-FullAWSAccess	FullAWSAccess	True
p-7n2kd4w9	deny-leaving-the-org	False
== ou-f6g7-8m3tw1ab
p-FullAWSAccess	FullAWSAccess	True
== 111122223333
p-FullAWSAccess	FullAWSAccess	True
$ aws organizations list-policies --filter SERVICE_CONTROL_POLICY --query 'Policies[].[Id,Name]' --output text
p-FullAWSAccess	FullAWSAccess
p-7n2kd4w9	deny-leaving-the-org
p-3h8xq5vz	workloads-region-guardrail
p-9c4tm1rk	sandbox-no-big-instances
$ aws organizations list-targets-for-policy --policy-id p-3h8xq5vz --query 'Targets[].[Name,Type]' --output text
Workloads	ORGANIZATIONAL_UNIT

FullAWSAccess at every level is what makes "SCPs allow everything" the default; the evaluation needs an allow at the root and at each OU and at the account. Detach FullAWSAccess from an OU without attaching another allow there, and everything below it is denied - even for admins, with "because no service control policy allows". AWS refuses to detach the last SCP of a target, but not one of two:

$ aws organizations detach-policy --policy-id p-FullAWSAccess --target-id 111122223333
aws: [ERROR]: An error occurred (ConstraintViolationException) when calling the DetachPolicy operation: You attempted to remove the last service control policy attached to the entity. An entity must have at least one service control policy attached.

Additional error details:
Reason: MIN_POLICY_TYPE_ATTACHMENT_LIMIT_EXCEEDED

Reading a guardrail

The Workloads OU limits its accounts to the two EU Regions:

$ aws organizations describe-policy --policy-id p-3h8xq5vz --query 'Policy.Content' --output text | jq '.Statement[0] | {Effect, Condition, exempt: (.NotAction | length), first: .NotAction[0:6]}'
{
  "Effect": "Deny",
  "Condition": {
    "StringNotEquals": {
      "aws:RequestedRegion": [
        "eu-central-1",
        "eu-west-1"
      ]
    },
    "ArnNotLike": {
      "aws:PrincipalArn": "arn:aws:iam::*:role/OrgBreakGlass"
    }
  },
  "exempt": 22,
  "first": [
    "iam:*",
    "sts:*",
    "organizations:*",
    "account:*",
    "cloudfront:*",
    "route53:*"
  ]
}

Three patterns are packed into it:

Writing one: deny, test, then attach

An SCP is the IAM policy language without Principal. Here, a deny on deleting S3 buckets, tried on this account first (attaching high in the tree is how one typo locks out a hundred accounts):

$ cat ~/oncall-lab/labs/aws/try3/deny-bucket-delete.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "NoBucketDeletes",
      "Effect": "Deny",
      "Action": [
        "s3:DeleteBucket",
        "s3:PutLifecycleConfiguration"
      ],
      "Resource": "*",
      "Condition": {
        "ArnNotLike": {
          "aws:PrincipalArn": "arn:aws:iam::*:role/OrgBreakGlass"
        }
      }
    }
  ]
}
$ P=$(aws organizations create-policy --type SERVICE_CONTROL_POLICY --name try-no-bucket-deletes --description "No S3 bucket deletes outside break-glass" --content file://$HOME/oncall-lab/labs/aws/try3/deny-bucket-delete.json --query Policy.PolicySummary.Id --output text)
$ aws organizations attach-policy --policy-id $P --target-id 111122223333
$ aws s3 mb s3://try-scp-test-111122223333 && aws s3 rb s3://try-scp-test-111122223333
make_bucket: try-scp-test-111122223333
remove_bucket failed: s3://try-scp-test-111122223333 An error occurred (AccessDenied) when calling the DeleteBucket operation: User: arn:aws:iam::111122223333:user/learner is not authorized to perform: s3:DeleteBucket on resource: "arn:aws:s3:::try-scp-test-111122223333" with an explicit deny in a service control policy, with policy ARN: arn:aws:organizations::444455556666:policy/o-a1b2c3d4e5/service_control_policy/p-50734b09
$ aws iam simulate-principal-policy --policy-source-arn arn:aws:iam::111122223333:user/learner --action-names s3:DeleteBucket --resource-arns arn:aws:s3:::try-scp-test-111122223333 --query 'EvaluationResults[0].[EvalDecision,OrganizationsDecisionDetail.AllowedByOrganizations]' --output text
explicitDeny	False

learner is an administrator, and the SCP still wins: the error names the policy type and its ARN, and the IAM simulator reports AllowedByOrganizations: False. Clean up:

$ aws organizations detach-policy --policy-id $P --target-id 111122223333 && aws organizations delete-policy --policy-id $P
$ aws s3 rb s3://try-scp-test-111122223333
remove_bucket: try-scp-test-111122223333

A policy cannot be deleted while it is attached (PolicyInUseException), at most 5 SCPs may be attached to one target, and one SCP may have 5,120 characters - which is why big organizations compress their policies with wildcards and NotAction.

What SCPs do not do

In an interview: "What is a service control policy and how is it different from an IAM policy?" - "An SCP is an Organizations guardrail on accounts or OUs: it never grants, it limits the maximum anyone in those accounts can do, the root user included, except in the management account. An action needs an allow from an SCP at every level of the path and no deny anywhere, and then still an IAM allow. A typical one denies everything outside approved Regions, with NotAction exemptions for global services like IAM and a break-glass role."

You can now: read an organization's tree and where an account sits in it, list the SCPs that apply to an account level by level, explain why FullAWSAccess is everywhere and what detaching it does, read a Region guardrail with its global-service exemptions, and write, test and remove an SCP without locking anyone out.

Why it helps

In a real company you will work inside an organization, and some of your AccessDenied errors will come from a guardrail above your account that no IAM change can fix. Reading the tree and the policies on your account's path tells you where a denial comes from and who owns it.

If you become the person who writes guardrails, the classic mistake - a Region restriction that also blocks the global services - can lock every account out of IAM or billing at once, so testing before attaching matters.

Commands in this lesson

aws cat

FAQ

Can an administrator in my account bypass an SCP?

No. An SCP limits every principal in the member accounts below it, the account's root user included. Only the management account is never limited by SCPs, which is why it should run nothing else and have very few people with access.

Why is FullAWSAccess attached everywhere?

For a request to be allowed, every level of the path - the root, each OU, the account - needs an SCP that allows it. AWS attaches FullAWSAccess to every root, OU and account by default; detaching it from one level without another allow there denies everything below.

Why did our Region guardrail break IAM and Cost Explorer?

Global services such as IAM, Organizations, Cost Explorer, Budgets, Route 53 and Support are called in us-east-1 whatever Region you work in. A deny of everything outside approved Regions must exempt them with NotAction, or every account under it loses them.

Can a member account see the SCPs that apply to it?

Normally not: Organizations calls other than describe-organization need the management account. A member account can be made a delegated administrator for policies, which lets it read the tree and manage SCPs - this lab account is one.

How do I test an SCP before attaching it widely?

Attach it to a test account or a sandbox OU first and run the calls your teams depend on. The IAM policy simulator evaluates SCPs too (OrganizationsDecisionDetail), and --context-entries lets you test conditions such as aws:RequestedRegion.

In an interview Mid

What is a service control policy, and how is it different from an IAM policy?

An SCP is an Organizations guardrail attached to the root, an OU or an account: it never grants, it limits the maximum anyone in those accounts can do, the root user included, except in the management account. An action needs an allow from an SCP at every level and no deny anywhere, and then still an IAM allow. A typical one denies everything outside approved Regions, with NotAction exemptions for global services like IAM.

Also asked: Why does an AWS organization have many accounts instead of one? · What happens if you detach FullAWSAccess from an OU? · How would you stop anyone from disabling CloudTrail in every account?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.