Skip to content
This repository was archived by the owner on Dec 31, 2023. It is now read-only.
This repository was archived by the owner on Dec 31, 2023. It is now read-only.

Connections stay in CLOSE_WAIT state on using kms_v1.KeyManagementServiceClient #2

Closed

Description

@rachari

On using kms_v1.KeyManagementServiceClient in one of the scripts that I'm working on, I noticed the following in lsof output for the process

 TCP <instance_ip>:42108->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42118->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42130->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42140->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42152->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42162->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42202->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42212->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42224->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42234->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42246->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42256->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42266->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42278->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42288->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42300->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42310->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42322->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42332->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42344->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42354->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42366->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42376->169.254.169.254:http (CLOSE_WAIT)

This happens irrespective of whether the KMS client is called in a loop or outside of a loop

If the script is being run as a scheduled job, then these CLOSE_WAIT sockets keep piling up eventually leading to a "Too many open files" error for the running process.

Is there a way to close these connections ?

NOTE -
Following packages are installed and being used :-

google-api-core                       1.15.0
google-api-python-client              1.7.11
google-auth                           1.6.3
google-auth-httplib2                  0.0.3
google-cloud-kms                      1.2.1
googleapis-common-protos              1.6.0
grpc-google-iam-v1                    0.12.3
grpcio                                1.26.0

Modified sample from https://cloud.google.com/kms/docs/reference/libraries (Python example) that can be used to check this :-

## Imports the Google APIs client library
import time
from google.cloud import kms_v1

# Your Google Cloud Platform project ID
project_id = 'YOUR_PROJECT_ID'

# Lists keys in the "global" location.
location = 'global'

# Creates an API client for the KMS API.
# client = kms_v1.KeyManagementServiceClient()

# The resource name of the location associated with the key rings.
# parent = client.location_path(project_id, location)

# Lists key rings
while True:
    # Creates an API client for the KMS API.
    client = kms_v1.KeyManagementServiceClient()

    # The resource name of the location associated with the key rings.
    parent = client.location_path(project_id, location)
    response = client.list_key_rings(parent, timeout=5)
    response_list = list(response)

    if len(response_list) > 0:
        print('Key rings:')
        for key_ring in response_list:
            print(key_ring.name)
    else:
        print('No key rings found.')

    time.sleep(30)

Activity

  1. added
    type: questionRequest for information or clarification. Not an issue.
    on Feb 5, 2020
  2. tswast commented on Aug 18, 2020

    @tswast

    This sounds a lot like issue googleapis/google-cloud-python#9457

    I've filed googleapis/gapic-generator-python#575 to track implementing closable client objects in the code generator.

  3. tswast commented on Aug 18, 2020

    @tswast

    In the meantime, I believe you'll need to call client._transport.channel.close() to clean up sockets.

  4. vchudnov-g commented on Jan 31, 2023

    @vchudnov-g

    Note that googleapis/gapic-generator-python#575 has been implemented, so it should be even easier to close connections.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    api: cloudkmsIssues related to the googleapis/python-kms API.type: questionRequest for information or clarification. Not an issue.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions