在django应用程序旁边持续运行任务

10 浏览
0 Comments

在django应用程序旁边持续运行任务

我正在构建一个Django应用程序,它根据特定算法列出热门的Twitter趋势话题。

我想要无限地运行一些进程,以便调用Twitter API并使用新信息更新数据库(postgre)。这样,热门趋势话题列表就可以异步更新。

起初,我认为celery+rabbitmq是解决我的问题的方法,但据我了解,它们是在Django内部用于启动定时或用户触发的任务,而不是无限运行的任务。

我脑海中的解决方案是编写一个.py文件,不断将热门话题放入队列,并编写独立的.py文件,不断运行,进行队列请求并使用原始SQL或SQLAlchemy将数据保存在Django使用的数据库中。我认为这个方法可以行得通,但我相信肯定有更好的方法来实现。

0
0 Comments

Running continually tasks alongside a Django app can be a challenge. However, there are solutions available to address this issue.

One common solution is to use Supervisor, a process control system. Supervisor allows you to keep processes running continuously, ensuring that your tasks are always running in the background. With Supervisor, you can easily manage and monitor these processes.

To implement this solution, you can combine Supervisor with any queuing technology of your choice. Queuing technologies such as RabbitMQ or Redis are popular choices for managing and pushing tasks into queues.

By using Supervisor in conjunction with a queuing technology, you can ensure that your tasks are continuously processed and executed alongside your Django app. This allows for efficient and seamless task management within your application.

Here is an example of how you can use Supervisor with a queuing technology:

1. Install Supervisor using pip:

pip install supervisor

2. Create a configuration file for Supervisor, typically named `supervisor.conf`. This file will define the processes you want to run continuously. Here is an example configuration:

[program:my_task]
command=/path/to/python /path/to/my_task.py
directory=/path/to/my_project
user=my_user
[program:my_queue_worker]
command=/path/to/python /path/to/my_queue_worker.py
directory=/path/to/my_project
user=my_user

In this example, we have defined two programs: `my_task` and `my_queue_worker`. These programs are executed using Python, and their respective scripts are located in the specified directories.

3. Start Supervisor and load the configuration file:

supervisord -c /path/to/supervisor.conf

4. Monitor the status of your processes using the Supervisor web interface. You can access the web interface by visiting the URL provided during the Supervisor startup.

By following these steps, you can ensure that your tasks and queue workers are always running alongside your Django app. Supervisor takes care of managing and monitoring these processes, allowing for seamless task execution.

In conclusion, running continually tasks alongside a Django app can be achieved by using Supervisor in combination with a queuing technology. This solution provides a reliable and efficient way to manage and execute tasks within your application.

0