Upgraded connectors causing workspace to be empty

Slack Knowledge Base Q&A Platform Questions
, , , ,
21

<@U07C8CCC68Y> if you don’t mind me asking about the --low-resource-mode setting: what kind of values do you see there and what kind of settings you tweaked in values.yml to make some performance gain?

22

The bulk of the changes can be found here: https://github.com/airbytehq/abctl/blob/main/internal/cmd/local/local/cmd.go#L357

23

I did not run the --migrate command, does that mean I lost all my 8 months of work?

24

No, an upgrade should be completely non-destructive. We do not delete any information unless you run an abctl local uninstall --persisted

25

can you run ls -lsa ~/.airbyte/abctl/data and paste the output here

26 
4 drwxr--r-- 4 simarpreet simarpreet 4096 Aug 16 18:02 .
4 drwxr--r-- 3 simarpreet simarpreet 4096 Aug 16 18:02 ..
4 drwxrwxrwx 6 simarpreet simarpreet 4096 Aug 16 18:05 airbyte-minio-pv
4 drwxrwxrwx 3 simarpreet simarpreet 4096 Aug 16 18:03 airbyte-volume-db```
27

Did you run an uninstall before running install?

28

It looks like those volumes were created today

29

nope I did not run uninstall

30

is it possible you were running abctl with a root account?

31

you could try ls -lsa /root/.airbyte/abctl/data to verify

32

ls: cannot access '/root/.airbyte/abctl/data': Permission denied

33

you will have to sudo su into your root account

34

ls: cannot access '/root/.airbyte/abctl/data': No such file or directory

35

were you using docker compose before or abctl?

36

Ah ok. In that case you should run abctl local install --migrate. You should run a docker compose stop if you have not done so. Do not run docker compose down as that could destroy your state. Documentation for the migration can be found here: https://docs.airbyte.com/using-airbyte/getting-started/oss-quickstart#migrating-from-docker-compose-optional

37

ah yes! that is where I made a mistake. Shouldn’t have run docker compose down

38

I think you might be ok. Try and run abctl local install with the --migrate command and :crossed_fingers: you get all your state back

39

<@U065RJ879QT> - Thanks for checking in! I tried changing the values to 100 records and 2 pages, then 50 records and 1 page, then 10 records and 1 page, still didn’t work and getting the same error. Do I also need to change Max number of partitions from 5 to a lower number?
Internal Server Error: com.fasterxml.jackson.databind.JsonMappingException: String value length (20054016) exceeds the maximum allowed (20000000, from `StreamReadConstraints.getMaxStringLength()`)

40

Also, the builder is ignoring all the keys that have null values in it. See the test response from builder vs what Postman pulls, this is causing downstream issues. What changed so much between docker-compose and abctl version of airbyte, that I never encountered such issues in the former airbyte version

{
    "InventoryID": "FREIGHT                       ",
    "Description": "Freight",
    "Weight": "0.000000",
    "CreatedBy": "admin admin",
    "CreatedOn": "2023-01-12T22:10:50.343",
    "BaseUnit": "EA",
    "PurchaseUnit": "EA",
    "SalesUnit": "EA",
    "ItemStatus": "No Sales",
    "IsaKit": false,
    "PostingClass": "FREIGHT",
    "MRPItem": "0"
  },

Postman result
{
            "InventoryID": "FREIGHT                       ",
            "Description": "Freight",
            "Weight": "0.000000",
            "CreatedBy": "admin admin",
            "CreatedOn": "2023-01-12T22:10:50.343",
            "BaseUnit": "EA",
            "PurchaseUnit": "EA",
            "SalesUnit": "EA",
            "ItemClass": null,
            "ItemStatus": "No Sales",
            "ItemTypeID": null,
            "IsaKit": false,
            "LotSerialClass": null,
            "PostingClass": "FREIGHT",
            "StrainCode": null,
            "WeightUOM": null,
            "DefaultBOMID": null,
            "FinishedGoodsWIPReference": null,
            "Dominance": null,
            "ItemFamilyGroup": null,
            "StrainFlavorLR": null,
            "Level1": null,
            "Level2": null,
            "Level3": null,
            "Level4": null,
            "RoshiItemID": null,
            "MRPItem": "0",
            "Merch_Bucket": null,
            "Merch_Category": null,
            "Merch_Color": null,
            "Merch_Kit": null,
            "Merch_Series": null,
            "Merch_Size": null,
            "Onboarding_Kit": null,
            "Swag_Tier": null
        },```