How to speed up import process in Store Manager for Magento

Magento 2 (Magento Open Source 2, Adobe Commerce) is highly stable eCommerce system, but when it comes to bulk operations like Magento 2 (Magento Open Source 2, Adobe Commerce) import, the procedure of upload can be really slow and take almost forever, making you guess whether it hanged or how much more you should wait. You what’s even worse? Finding the reason of this delay. In most cases it is much more difficult task than actions aimed at improving of import performance optimization.

Generally, Magento 2 (Magento Open Source 2, Adobe Commerce) import speed depends upon server resources, settings you have and number of goods existing at your store. Speaking of resources and limitations, they should be enough to handle operations you perform - by that is meant that server with high performance and extended memory limit will contribute greatly to overall performance of the procedure. In terms of settings, you might find useful for example enabling automatic logs clearing. But be careful, since in case you will need to check logs because something went wrong, you might not find necessary ones, so you may copy them from time to time (or at least before mass operations). Finally, the more goods you already have, the more records are in your database.

Respectively, the larger it becomes, the heavier is input or update of products.

Luckily, there are a few techniques to speed the procedure up. We will not ask you to delete or clear some database tables, which by the way, is potentially unsafe unless you are 100% sure what you are doing. Separating the files in portions and import in chunks can be way-out, but think - you will have to run it a few times, so what’s the point?

Slow import to Magento 2 (Magento Open Source 2, Adobe Commerce) can be optimized in performance by using more flexible tools for data upload, like Store Manager for Magento 2 (Magento Open Source 2, Adobe Commerce). This useful tool allows you to work directly from your PC. Moreover, with bridge type of database connection you will be able to work with the local copy of your database and then only post changes to your live website. This way, there will be the less load on your server, comparing to direct changes, at which each time requests to server are made.

In order to speed up the import process you need to check the following options at the last step of Import Wizard:

• Select "Add Only" option in case you want to update your store only with new products without changing the existing ones.

  When this option is used, import procedure will not check if product already exists but will always add new one. If you perform import from the same file again and again it will create new products each time so there will be duplicated entries. This option should be used only for initial import when you just starting new store. It should NOT be used for regular updates such as automated import.

• Check "Cache import data" checkbox that will help to speed up the process if you have slow direct connection.

  This option may boost performance when importing from large .csv files (approx. 5000 products and more). For smaller files it may not be effective. Also the more products you already have at your store the more time it will take to create local product cache so if you have 100000 or more products in your store database and want to import about 5000 products from .csv file this option may not be so effective. This option has no effect at all when you're using bridge connection.

• Check "Accelerate import by increasing memory usage" checkbox.

  It is recommended to have this option always checked unless you have really low amount of memory available at your system. However, in some cases this option may have the opposite effect. It does retrieve ALL products from your store and creates memory maps allowing import procedure to determine faster whether current product is new or already exists. In case if you have large database (100 000 products and more) it may take too long to retrieve all of them and will take too much memory as well. This option is most effective in case of you have up to 10 000 products in your database and want to import a few more thousands.

• Uncheck "Reindex product data" checkbox and make full Reindex data in the Magento 2 (Magento Open Source 2, Adobe Commerce) admin or in Store Manager after the import is complete.

  Product import procedure consists of two major parts - product import itself and product indexing. Indexing takes about 60-70% of total import time so when this option is unchecked it may be significantly faster. Nevertheless product indexing is VERY important in Magento 2 (Magento Open Source 2, Adobe Commerce). When index is incorrect product may not be visible at your store or may cause various errors. You have to rebuild product index in any case when import is finished. In some cases it may take less time to do that in Magento 2 (Magento Open Source 2, Adobe Commerce) admin area: System -> Index Management.

Please check the screen-capture of all the steps you have to perform:



• Important images settings: Magento 2 (Magento Open Source 2, Adobe Commerce) requires unique names for all product images. For these purposes, an option to generate a new image name if the same name already exists in a database was added in Store Manager. However, since this option requires Store Manager to scan all folders containing images at your FTP server, import procedure may take more time than expected. If you completely sure that your image names are unique you can uncheck this option to speed up the upload.

Other things that may affect import speed:

• Number of Store Views you have checked to import products to. In general, every Store View has its own set of product attributes. If you select two store views import procedure will actually post data twice and respectively three times for three selected Store Views and so on.
• Number of product attributes in attribute set. The more attributes product has the slower import will be. Do not create product attributes that you do not need and delete those ones you're not using. If product attribute has "Use in Layered Navigation" option set to "Yes" it will be included in indexing and thus slow down import and other operations with a product even more.
• Number of .csv fields assigned to product database fields i.e. number of columns you want to import from a .csv file to your store. The more columns you have linked the more time it will take to import a product. The number of assigned columns also affects product indexing.
• The number of products you already have in your database. Even if you want to import 10 products but already have one million of them in the store database it will take a long time. Import has to perform several checks for every product and so it might be slow. If you're starting with an empty database, at the beginning Store Manager may import 20-30 or even more products per second, however, every new product added to the database will slow down import speed so it may drop to 1 product per second and that is NORMAL.
• Database server performance. Store Manager sends SQL commands to the server. It may take some time for the server to perform these commands. The more CPU and memory resources available at the server the faster it will execute requested command and the faster it will work in general. Do not expect high performance if you're using shared public hosting - usually, they provide resources for simple HTML websites but not for e-Commerce engines like Magento 2 (Magento Open Source 2, Adobe Commerce).

Generally, all you have to remember - large amount of data will always take large amount of time to be processed.