Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
Hubspot Extract to PubSub
Hubspot Extract to PubSub
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 1
    • Issues 1
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
    • Iterations
  • Merge requests 0
    • Merge requests 0
  • Requirements
    • Requirements
    • List
  • Operations
    • Operations
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Package Registry
    • Container Registry
  • Analytics
    • Analytics
    • Code Review
    • Insights
    • Issue
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
Collapse sidebar
  • Cosmos
  • Modules
  • Hubspot Extract to PubSubHubspot Extract to PubSub
  • Issues
  • #6

Closed
Open
Created Oct 28, 2020 by Ray Walker@rawalkerOwner

Identify cause, fix PubSub batch publish random segfaults

On my system I can only trigger execution between one and five times before I get a segfault. Can @all confirm / deny?

╰─ make dev
fatal: No names found, cannot describe anything.
cd src && functions-framework --target=main --signature-type=event
BUCKET_NAME: cosmos-dev-286703-hs-extract-gcs-cosmos-dev-temp
Fetching secret: cosmos-dev-286703/hubspot_api_key_cosmos_dev/versions/latest ...
[2020-10-28 14:56:38 +1100] [26924] [INFO] Starting gunicorn 20.0.4
[2020-10-28 14:56:38 +1100] [26924] [INFO] Listening at: http://0.0.0.0:8080 (26924)
[2020-10-28 14:56:38 +1100] [26924] [INFO] Using worker: threads
[2020-10-28 14:56:38 +1100] [26967] [INFO] Booting worker with pid: 26967
Hubspot Api Get Email Events
End: 1999-12-31 13:00:00
Start: 2020-10-28 03:56:40
# events: 1000
> hasMore: True
Published 1000 messages to projects/cosmos-dev-286703/topics/hs-test-email-events.
/home/fish/gpi/projects/ged/cosmos/modules/cosmos_hubspot_extract_gcs/src/main.py executed in 1.27 seconds.
Hubspot Api Get Email Events
End: 1999-12-31 13:00:00
Start: 2020-10-28 03:56:54
# events: 1000
> hasMore: True
Published 1000 messages to projects/cosmos-dev-286703/topics/hs-test-email-events.
/home/fish/gpi/projects/ged/cosmos/modules/cosmos_hubspot_extract_gcs/src/main.py executed in 0.75 seconds.
Hubspot Api Get Email Events
End: 1999-12-31 13:00:00
Start: 2020-10-28 03:57:01
# events: 1000
> hasMore: True
Published 1000 messages to projects/cosmos-dev-286703/topics/hs-test-email-events.
/home/fish/gpi/projects/ged/cosmos/modules/cosmos_hubspot_extract_gcs/src/main.py executed in 1.50 seconds.
Hubspot Api Get Email Events
End: 1999-12-31 13:00:00
Start: 2020-10-28 03:57:07
# events: 1000
> hasMore: True
Published 1000 messages to projects/cosmos-dev-286703/topics/hs-test-email-events.
/home/fish/gpi/projects/ged/cosmos/modules/cosmos_hubspot_extract_gcs/src/main.py executed in 0.73 seconds.
Hubspot Api Get Email Events
End: 1999-12-31 13:00:00
Start: 2020-10-28 03:57:11
# events: 1000
> hasMore: True
Published 1000 messages to projects/cosmos-dev-286703/topics/hs-test-email-events.
/home/fish/gpi/projects/ged/cosmos/modules/cosmos_hubspot_extract_gcs/src/main.py executed in 1.36 seconds.
make: *** [Makefile:61: dev] Segmentation fault
[2020-10-28 14:57:13 +1100] [26967] [INFO] Parent changed, shutting down: <Worker 26967>
[2020-10-28 14:57:13 +1100] [26967] [INFO] Worker exiting (pid: 26967)
# pip freeze -r src/requirements.txt
google-cloud-pubsub==2.1.0
google-cloud-secret-manager==1.0.0
## The following requirements were added by pip freeze:
aiostream==0.4.1
appdirs==1.4.4
astroid==2.4.2
attrs==20.2.0
black==20.8b1
cachetools==4.1.1
cbor2==5.1.2
certifi==2020.6.20
cffi==1.14.2
cfgv==3.2.0
chardet==3.0.4
civis==1.14.2
click==7.1.2
cloudevents==0.3.0
cloudpickle==1.6.0
distlib==0.3.1
filelock==3.0.12
fire==0.1.3
Flask==1.1.2
functions-framework==2.0.0
google-api-core==1.22.2
google-api-python-client==1.12.2
google-auth==1.21.1
google-auth-httplib2==0.0.4
google-cloud-bigquery==1.28.0
google-cloud-core==1.4.1
google-cloud-storage==1.32.0
google-crc32c==1.0.0
google-resumable-media==1.0.0
googleapis-common-protos==1.52.0
grpc-google-iam-v1==0.12.3
grpcio==1.32.0
gunicorn==20.0.4
httplib2==0.18.1
hubspot-api-client==3.1.0
hubspot3==3.2.50
identify==1.5.0
idna==2.10
importlib-metadata==1.7.0
isort==5.5.1
itsdangerous==1.1.0
Jinja2==2.11.2
joblib==0.14.1
jsonref==0.2
jsonschema==3.2.0
lazy-object-proxy==1.4.3
libcst==0.3.10
livereload==2.6.3
MarkupSafe==1.1.1
mccabe==0.6.1
mypy-extensions==0.4.3
nodeenv==1.5.0
numpy==1.19.1
pandas==1.1.2
pathspec==0.8.0
pathtools==0.1.2
pre-commit==2.7.1
proto-plus==1.9.1
protobuf==3.13.0
pubnub==4.5.3
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
pycryptodomex==3.9.8
pylint==2.6.0
pyrsistent==0.16.0
python-dateutil==2.8.1
python-gitlab==2.5.0
pytz==2020.1
PyYAML==5.3.1
regex==2020.7.14
requests==2.24.0
rsa==4.6
sentry-sdk==0.19.1
six==1.15.0
toml==0.10.1
tornado==6.0.4
typed-ast==1.4.1
typing-extensions==3.7.4.3
typing-inspect==0.6.0
uritemplate==3.0.1
urllib3==1.25.10
virtualenv==20.0.31
watchdog==0.10.3
Werkzeug==1.0.1
wrapt==1.12.1
zipp==3.1.0
Assignee
Assign to
Sprint 1
Milestone
Sprint 1 (Past due)
Assign milestone
Time tracking