With multi-dimensional model as the core, let the factory digital transformation and upgrading “within reach”>>>

Due to an error in the title:

Message: Tried to run command without establishing a connection

Explanation:

Let’s talk about my crawler architecture, which uses firebox + selenium. The error above is that after the browser exits, the crawler still needs the browser to execute some commands, so it reports the above error. Here’s the second problem:

Why does the browser crash automatically?Generally speaking, it is because the resources are not enough. What resources?Memory resources. The browser takes up a lot of memory. In order to speed up the crawler, some crawlers make the browser cache

This causes the browser to use more memory

Solution:

1. According to the resources occupied by the crawler, increase the memory appropriately

2. Reduce the speed of the crawler to make the browser a little idle, especially when starting multiple browsers

———————————Updated on April 24, 2019——————————–

There is nothing wrong with the above statement, but there are still some cases that are not taken into account. In fact, the main reason why browsers will crash is because of memory leaks in browsers. That is to say, because browsers turn on caching by default

In this way, with the operation of the crawler, the browser cache will become larger and larger, which will eventually lead to the memory leakage of the browser (if the browser does not restart the crawler, if the browser restarts after a period of time, there will be no such problem)

As for how to disable browser caching in crawler, I mentioned it in another blog, so I won’t talk about it