Import/Export API Overview

The Import/Export tool can be used through the API as well as through the user interface in the KCCP Admin. Using the APIs you can write integrations that can upload data into KCCP at blazing speed. Any bulk data load into KCCP should use the Import/Export APIs if the data resource is supported.

These APIs are used internally by the Import/Export 3.0 UI, so you can use the UI to export files which can then be modified and imported by the API. Similarly, you can create an export via API, and import it in the Import/Export 3.0 UI.

Overview

The import and export APIs use a series of API calls to coordinate the process of getting data in and out of Kibo. File upload and download is handled separately than the creation of the jobs themselves. The API calls for the jobs are asynchronous in that none of the API calls will block for the jobs to complete, you must poll for the results.

The following diagrams show the general process of the series of API calls needed for both export and import:

Export Process

The export process is as follows:

  1. Form a JSON payload describing the export that you want to perform, then POST it to the Export Create API. 
  2. The Export Create API returns an ID which is used to track the export process.
  3. Poll the Export Get API until you see that the isComplete field is true. (typically every few seconds) When it is true, inside the fileskey in the JSON there will be an object where "fileType": "export", then take that ID.
  4. Use the POST Files Get Public Link API with that ID, which gives you the contents of the export.

Import Process

The import process is as follows:

  1. Create a ZIP file containing the CSV files you want to import. Then POST it to the Files Upload API. This returns an ID.
  2. Call the Import Create API, referencing the file that you uploaded in the previous step. You will then receive an ID that you can use to track the import.
  3. Poll the Import Get API until you see that the isComplete field is true. Once this field returns as true, then the import is complete.

A First Example in Postman

First, make sure that you have the autogenerated Postman collection set up. See Getting Started with your API for details.

  1. POST this payload to the Export Create API:
    {
  "name": "Export",
  "domain": "catalog",
  "resources": [
    {
      "resource": "Products",
      "format": "legacy"
    }
  ],
  "format": "legacy"
}
  2. Call the Export List API to see all of the recent exports. Refresh until isComplete is true. You will receive a result like this:
    [
    {
        "name": "Export",
        "id": "12f69402-e8ea-44d4-b2b5-b2d5fe5e9f27",
        "requester": "9154becb518b42ffb5d0a82519abd75a",
        "tenant": 39668,
        "domain": "catalog",
        "resources": [
            {
                "resource": "Products",
                "format": "legacy",
                "status": "complete",
                "isComplete": true,
                "stateDetails": "Duration: 0.0390942 seconds"
            }
        ],
        "format": "legacy",
        "status": "complete",
        "isComplete": true,
        "files": [
            {
                "id": "cbcfbbfb-bc4c-4132-ab0a-8a6fe1e2a9cb",
                "locationType": "internal",
                "fileName": "catalog_export.zip",
                "fileType": "export"
            }
        ],
        "auditInfo": {
            "updateDate": "2023-09-20T14:26:13.887Z",
            "createDate": "2023-09-20T14:26:11.327Z"
        }
    }
]
  3. Then call Files Get Public Link on the ID of the export, in this case cbcfbbfb-bc4c-4132-ab0a-8a6fe1e2a9cThis will return a pre-signed URL which you can paste in a web browser to retrieve the product data of the tenant that you just exported.
    "https://example.kibocommerce.com/stg1/v3/29594/files/522675d7-a2df-4c4a-a38f-189b383f4c0b.zip?AWSAccessKeyId=ASIA32EK6I523JDJSM7T&Expires=1695239594&x-amz-security-token=IQoJb3JpZ2luX2VjENv%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJIMEYCIQDSANS1V%2Ffq1zTEo7jje%2FpbhnsBa3DROwPkPYqqn3ibKgIhAMTRqGfdRtJ3uaVV51V6sdpAty8yNBAivRAMcFkSoBgNKroFCMT%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEQBBoMODEyMDQwMjEwMjkzIgwEWdXqLCb1sInw%2FtYqjgWPb5V3u5fzpjBwEUQvgck5lGnVwyPoLC0p4N7CIhXWDZPSrYzlcuKI5Z%2Br%2FqG0vmjOt9tuzgEm%2F%2FpzAQHMjtAPyaTYn8zlwfJ7DYOCq9Hhdh6QRyatyCux1C9Yg8yyBXzilVNxL%2BT4f%2BbSYHpdhZeDaKOtcO9jp%2FWO0ca%2FBEgj6Wr6BXPCmGrc%2BOsLSymA6quW6fA19462DNsbWyqC13Miv%2Bh4QIIAFTSnvWKEey6ZNbEh8%2FBSXKw9mXnlmgL8ITu8dL9lxRJpawyh1N3udI75IKWHjNfP2tZo0%2FpgDDwlsoaFBXyV%2FiAnrnzm9OiJsCjvvrxxGiLN2jqNY%2BNA8bh%2BNGXwo4n5uii%2BS7D671Pq2IvIJzemdomPBjeEkNZBi3HHi3gtVJqHwGLDkCe1jWFShEbWlt7t7nkvOagDiZibdmtNPr6CtVbXgLyk3ilujBrMlxdK8I1K%2BeKEDXQAgSyqEFhNVxx%2B4suUelxbBlVG0SbHp74Ey5ODWrm6wBoU5SV0MtRGptF20bIKeMcmjjL5T1BBvyP%2BHiPtSrtblMOK4ZUAe9laJQ%2BWtPsJ3SsGNPMtfp4P%2Bswi%2BPRhN4H7IP9e%2BFJgu4gmMyae3ZZF%2BQ%2BOmlW12LnEBRCyle7Y%2F3oCzio0RVbO8YySuj%2B9%2FiU%2B8Nqy6%2BcMOfgVAjFXTkj0a2vzh5biBu8bDWids8V8u71bji6YSpxrTQknu%2B2MYRf9xGnigV2Fdaw5Un4GKUKEH8JSuDwz%2F6eUO7Fau0qsvVnDOHwcHDFfN5Q90Xt5CcFqiQNasZhHPfdazUop%2BKCVi%2BeKiUOFFp4n62Vc7HTEGZWHzIgUCnHkQamv7ESwhcZ5A8HdDBUOMZpBacloMSW%2BTb0w792nqAY6sAFXaVtQhcMY%2BJOSeNdZQAiKK3X5AfbrzAvbz6DQGbtWKwV9YIqrSxksmQ0MXfBICNvI4nrP%2Fl1z9uKEMzZpZ8dGGnVx9D541uHPN9frpffH%2BlZ4rboysZRw%2BWqveuH%2BlUXuRclzrMmURK%2BvrnbRhSFeWfg2Tei0rMUAx37XI%2BHc8GazbfMRQI1JVjw2H79dMy2cvRAr3eZ%2BUtx0%2FtB%2F5FpBztPSYQ7U0QBd3P%2BALc%2BuFg%3D%3D&Signature=sRO5TPhkLor09zbdnz8m7LqMrhQ"

CSV File format

The format of the CSV files follows the standard conventions for CSV file formats.

  • The file encoding should be UTF-8.
  • Use double quotes to wrap single quotes.
  • Quotes capture new lines.
  • To represent multiple values in a single field, separate by carriage returns (0xD in ASCII).

Be careful when opening up CSV files in Excel.  Excel can corrupt data (such as large numbers being converted to scientific notation). Best practice would be to open CSV in raw text format or if opened in Excel, do not save the document. Please check:

  • Accidental removal of leading zeros
  • Numbers that can be represented by scientific notation

deleteOmitted field

The resources field of the Create Import API contains a field deleteOmitted. By default, the import process will not delete any Kibo data if that data does not exist in the import. So, for example, if your import file contains only product 1001, and you already have products 1001 and 1002, 1002 will remain unchanged.

The deleteOmitted flag lets you specify that you have a full data import, and to wipe any data that does not exist as part of the CSV file for that resource. This lets you specify full updates. 

Removing Product Properties

By default, if you leave a product property blank, it will not clear the property in KCCP to prevent deletion of data. If you really need to clear a product property, use the ~delete~ sentinel as the value of the property to remove the property in the product in KCCP.

Exporting Optional Fields

Note that by default, all fields of resource are not exported. So for example, there is a Description field on the Attribute resource, but it is not exported by default, you must include it in the fields array. The full list of optional fields is specified in the Field List JSON.

Using Context Override

You can use the contextOverride field in an import to specify which catalogs or sites the data should be uploaded to:

"contextOverride": {
      "locale": "en-US",
      "currency": "USD",
      "masterCatalog": 2,
      "catalog": 5,
      "site": 54321 
}

Note that this field is required for ProductImages upload.

End of Central Directory Issues

If you ever receive errors about “End of Central directory”, this is most likely related to the file that was uploaded. It must be of zip file format, even if there is a single file. So for example, if you only want to update the Products resource, you need to take your file products.csv, zip it, and then upload the resulting zip file.

Example file structure for a single resource:

catalog.zip
  - products.csv

Product Images

In order to generate a ProductImages import file, it is recommended that as part of the image import process, a mapping is generated between the `cms-id` generated from upload and the product code it should apply to. This association can be used to generate an import CSV.

productimages.zip

Field List Specification

The following is a full JSON listing of the fields, which can be used for reference for what resources are available, and for those resources, what fields are required and optional.

{
  "Attributes": {
    "format": "Legacy",
    "resource": "Attributes",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "AttributeCode",
        "AttributeName",
        "DataType",
        "InputType",
        "IsExtra",
        "IsOption",
        "IsProperty",
        "Namespace",
        "SearchableInStorefront",
        "SearchableInAdmin",
        "SearchDisplayValue"
      ],
      "optional": [
        "Description"
      ]
    }
  },
  "AttributeValues": {
    "format": "Legacy",
    "resource": "AttributeValues",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "AttributeCode",
        "DataType",
        "Namespace",
        "Value",
        "Name",
        "DisplayOrder"
      ],
      "optional": []
    }
  },
  "ProductTypes": {
    "format": "Legacy",
    "resource": "ProductTypes",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "ProductType",
        "Standard",
        "Configurable",
        "Bundle",
        "Component"
      ],
      "optional": [
        "GoodsType"
      ]
    }
  },
  "ProductTypeAttributes": {
    "format": "Legacy",
    "resource": "ProductTypeAttributes",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "ProductType",
        "AttributeCode",
        "Type",
        "Order"
      ],
      "optional": [
        "IsRequiredByAdmin",
        "IsHiddenProperty",
        "IsMultiValueProperty"
      ]
    }
  },
  "ProductTypeAttributeValues": {
    "format": "Legacy",
    "resource": "ProductTypeAttributeValues",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "ProductType",
        "AttributeCode",
        "Type",
        "VocabularyValue"
      ],
      "optional": []
    }
  },
  "Categories": {
    "format": "Legacy",
    "resource": "Categories",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "CatalogName",
        "CategoryCode",
        "CategoryName",
        "CategoryType"
      ],
      "optional": [
        "CategoryId",
        "ParentCategoryCode",
        "Expression",
        "IsDisplayed",
        "IsActive",
        "Sequence",
        "MetaTagTitle",
        "MetaTagDescription",
        "PageTitle",
        "CategoryDescription",
        "MetaTagKeyWords",
        "SEOUrl"
      ]
    }
  },
  "CategoryImages": {
    "format": "Legacy",
    "resource": "CategoryImages",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "CatalogName",
        "CategoryCode",
        "ImageName"
      ],
      "optional": [
        "LocaleCode",
        "AltText",
        "ImageLabel",
        "Order"
      ]
    }
  },
  "SortDefinitions": {
    "format": "Legacy",
    "resource": "SortDefinitions",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "CatalogName",
        "CategoryCode",
        "ProductSortDefinitionId",
        "Name"
      ],
      "optional": [
        "StartDate",
        "EndDate",
        "PrimaryAttribute",
        "PrimaryDirection",
        "SecondaryAttribute",
        "SecondaryDirection"
      ]
    }
  },
  "ProductRankings": {
    "format": "Legacy",
    "resource": "ProductRankings",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "CatalogName",
        "CategoryCode",
        "SortDefinitionId",
        "ProductCode",
        "Position"
      ],
      "optional": [
        "Locked"
      ]
    }
  },
  "Products": {
    "format": "Legacy",
    "resource": "Products",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "ProductCode",
        "ProductType",
        "ProductUsage",
        "ProductName",
        "Price",
        "VariationPricingMethod",
        "PackageWeight",
        "PackageLength",
        "PackageWidth",
        "PackageHeight"
      ],
      "optional": [
        "SalePrice",
        "Cost",
        "MSRP",
        "MAP",
        "MAPEffectiveStartDate",
        "MAPEffectiveEndDate",
        "RestrictDiscount",
        "RestrictDiscountStartDate",
        "RestrictDiscountEndDate",
        "ManufacturerPartNumber",
        "UPC",
        "DistributorPartNumber",
        "IsTaxable",
        "ManageStock",
        "IsPackagedStandAlone",
        "OutOfStockBehavior",
        "FulfillmentTypes",
        "ProductShortDescription",
        "ContentFullProductDescription",
        "ContentFullProductDescription2",
        "ContentFullProductDescription3",
        "SEOMetaTagTitle",
        "SEOMetaTagDescription",
        "SEOMetaTagKeywords",
        "SEOFriendlyURL"
      ]
    }
  },
  "ProductPropertyLocale": {
    "format": "Legacy",
    "resource": "ProductPropertyLocale",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "ProductCode",
        "AttributeName"
      ],
      "optional": [
        "Value"
      ]
    }
  },
  "ProductCatalog": {
    "format": "Legacy",
    "resource": "ProductCatalog",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "CatalogName",
        "ProductCode",
        "IsActive"
      ],
      "optional": [
        "CategoryCodes",
        "ProductName",
        "Price",
        "SalePrice",
        "MSRP",
        "MAP",
        "MAPEffectiveStartDate",
        "MAPEffectiveEndDate",
        "ProductShortDescription",
        "ContentFullProductDescription",
        "SEOMetaTagTitle",
        "SEOMetaTagDescription",
        "SEOMetaTagKeywords",
        "SEOFriendlyURL"
      ]
    }
  },
  "ProductBundles": {
    "format": "Legacy",
    "resource": "ProductBundles",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "ProductCode",
        "Code",
        "Quantity"
      ],
      "optional": [
        "Name"
      ]
    }
  },
  "ProductOptions": {
    "format": "Legacy",
    "resource": "ProductOptions",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "ProductCode",
        "VariationCode",
        "Enabled"
      ],
      "optional": [
        "ExtraPrice",
        "FixedListPrice",
        "FixedSalePrice",
        "FixedMSRP",
        "DeltaMSRP",
        "FixedWeight",
        "ExtraWeight",
        "ExtraCost",
        "FulfillmentTypes",
        "MfgPartNo",
        "UPC",
        "DistPartNo"
      ]
    }
  },
  "ProductExtras": {
    "format": "Legacy",
    "resource": "ProductExtras",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "ProductCode",
        "AttributeCode",
        "Value"
      ],
      "optional": [
        "RequiredByShopper",
        "Defaulted",
        "MultiSelect",
        "Quantity"
      ]
    }
  },
  "ProductOptionsLocale": {
    "format": "Legacy",
    "resource": "ProductOptionsLocale",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "ProductCode",
        "VariationCode"
      ],
      "optional": [
        "Currency",
        "Extra Price",
        "Extra MSRP",
        "Extra Credit Value",
        "Fixed List Price",
        "Fixed Sale Price"
      ]
    }
  },
  "ProductImages": {
    "format": "Legacy",
    "resource": "ProductImages",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "ProductCode",
        "Name"
      ],
      "optional": [
        "Sequence",
        "AltText",
        "ImageLabel",
        "VideoUrl"
      ]
    }
  },
  "LocationTypes": {
    "format": "Legacy",
    "resource": "LocationTypes",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "LocationTypeCode",
        "LocationTypeName"
      ],
      "optional": []
    }
  },
  "Locations": {
    "format": "Legacy",
    "resource": "Locations",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "LocationTypeCodes",
        "LocationCode",
        "Name",
        "IsDisabled",
        "Address1",
        "AddressType",
        "CityOrTown",
        "CountryCode",
        "PostalOrZipCode",
        "StateOrProvince",
        "Shipping Contact CompanyOrOrganization",
        "Shipping Contact PhoneNumber",
        "SupportsInventory",
        "AllowFulfillmentWithNoStock",
        "Latitude",
        "Longitude",
        "Express",
        "TransferEnabled",
        "IncludeInLocationExport",
        "IncludeInInventoryAggregrate",
        "WarehouseEnabled"
      ],
      "optional": [
        "Description",
        "FulfillmentTypes",
        "Address2",
        "Address3",
        "Address4",
        "Shipping Contact Email",
        "Shipping Contact FirstName",
        "Shipping Contact LastNameOrSurname",
        "Shipping Contact MiddleNameOrInitial",
        "Fax",
        "Phone",
        "Notes",
        "Tags",
        "Hours of operation - Sunday",
        "Hours of operation - Monday",
        "Hours of operation - Tuesday",
        "Hours of operation - Wednesday",
        "Hours of operation - Thursday",
        "Hours of operation - Friday",
        "Hours of operation - Saturday",
        "LastModifiedDate"
      ]
    }
  },
  "LocationInventory": {
    "format": "Legacy",
    "resource": "LocationInventory",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "ProductCode",
        "LocationCode"
      ],
      "optional": [
        "ParentProductCode",
        "StockOnHand",
        "StockUpdateOption",
        "StockOnBackOrder",
        "StockAvailable"
      ]
    }
  },
  "Images": {
    "format": "Legacy",
    "resource": "Images",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "ImageUrl"
      ],
      "optional": [
        "Name",
        "Id",
        "Tags"
      ]
    }
  },
  "LocationGroup": {
    "format": "Legacy",
    "resource": "LocationGroup",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "LocationGroupCode",
        "Name",
        "LocationCodes",
        "SiteIds"
      ],
      "optional": [
        "LastModifiedDate"
      ]
    }
  },
  "LocationGroupConfiguration": {
    "format": "Legacy",
    "resource": "LocationGroupConfiguration",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "LocationGroupCode",
        "SiteId",
        "CustomerFailedToPickupAfterAction",
        "CustomerFailedToPickupDeadline",
        "SendCustomerPickupReminder",
        "DefaultCarrier",
        "PrintReturnLabel",
        "DefaultPrinterType"
      ],
      "optional": [
        "LastModifiedDate"
      ]
    }
  },
  "LocationGroupBoxTypeConfig": {
    "format": "Legacy",
    "resource": "LocationGroupBoxTypeConfig",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "LocationGroupCode",
        "SiteId",
        "Name",
        "Height",
        "Length",
        "Width"
      ],
      "optional": []
    }
  },
  "LocationGroupCarrierConfig": {
    "format": "Legacy",
    "resource": "LocationGroupCarrierConfig",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "LocationGroupCode",
        "SiteId",
        "CarrierType",
        "IsEnabled",
        "EnableSmartPost",
        "Express1DayDefault",
        "Express2DayDefault",
        "Express3DayDefault",
        "ReturnLabelShippingMethod",
        "ShippingMethods",
        "StandardDefault"
      ],
      "optional": []
    }
  },
  "Pricelists": {
    "format": "Legacy",
    "resource": "Pricelists",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "PriceListName",
        "PriceListCode",
        "Enabled",
        "ValidForAllSites",
        "ValidSites"
      ],
      "optional": [
        "ParentPriceListCode",
        "Description",
        "FilteredInStorefront",
        "IndexedSites",
        "DefaultForSites",
        "MappedCustomerSegments",
        "Resolvable",
        "ResolutionRank",
        "CreatedBy"
      ]
    }
  },
  "PricelistEntries": {
    "format": "Legacy",
    "resource": "PricelistEntries",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "PriceListCode",
        "ProductCode",
        "CurrencyCode",
        "StartDate",
        "IsVariation"
      ],
      "optional": [
        "ProductName",
        "OptionSummary",
        "EndDate",
        "PriceListEntryMode",
        "MsrpMode",
        "Msrp",
        "CostMode",
        "Cost",
        "MapMode",
        "Map",
        "MapStartDate",
        "MapEndDate",
        "DiscountsRestricted",
        "DiscountsRestrictedMode",
        "DiscountsRestrictedStartDate",
        "DiscountsRestrictedEndDate",
        "CreatedBy",
        "PriceListEntryTypeCode"
      ]
    }
  },
  "PricelistEntryPrices": {
    "format": "Legacy",
    "resource": "PricelistEntryPrices",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "PriceListCode",
        "ProductCode",
        "CurrencyCode",
        "StartDate",
        "MinimumQuantity"
      ],
      "optional": [
        "ListPriceMode",
        "SalePriceMode",
        "ListPrice",
        "SalePrice"
      ]
    }
  },
  "PricelistEntryExtras": {
    "format": "Legacy",
    "resource": "PricelistEntryExtras",
    "deleteOmitted": false,
    "configuration": {},
    "fields": {
      "required": [
        "MasterCatalogName",
        "PriceListCode",
        "ProductCode",
        "CurrencyCode",
        "StartDate",
        "AttributeFQN",
        "Value",
        "Price"
      ],
      "optional": [
        "DisplayValue",
        "AttributeCode"
      ]
    }
  }
}

Sample Data

sample.zip

Related Articles

How would you rate this article?
Thank you for your feedback!