lots of process for message queue …

Hello

Orocommerce 1.10, ubuntu 16, php 7

i have lots of process oro:message-queue:consume
and many logs that show problem with them

many lines with

Shell

[2017-04-21 22:21:07] console.ERROR: An error occurred while running command "'oro:message-queue:consume' --env=prod". Unable to launch a new process. {"exit_code":0,"arguments":{"command":"oro:message-queue:consume"},"options":{"env":"prod"}} [] [2017-04-21 22:29:03] app.ERROR: Consume messages command exception. "Unable to launch a new process." {"exception":"[object] (Symfony\\Component\\Process\\Exception\\RuntimeException(code: 0): Unable to launch a new process. at /srv/orocommerce/orocommerce-application/vendor/symfony/symfony/src/Symfony/Component/Process/Process.php:293)"} []

1 2

[2017-04-21 22:21:07] console.ERROR: An error occurred while running command "'oro:message-queue:consume' --env=prod". Unable to launch a new process. {"exit_code":0,"arguments":{"command":"oro:message-queue:consume"},"options":{"env":"prod"}} [] [2017-04-21 22:29:03] app.ERROR: Consume messages command exception. "Unable to launch a new process." {"exception":"[object] (Symfony\\Component\\Process\\Exception\\RuntimeException(code: 0): Unable to launch a new process. at /srv/orocommerce/orocommerce-application/vendor/symfony/symfony/src/Symfony/Component/Process/Process.php:293)"} []

and some processes are :

Shell

orocomm+ 21755 0.0 0.0 4508 612 ? Ss 21:39 0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 21759 0.1 2.8 389248 111772 ? S 21:39 0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 21807 0.0 0.0 4508 608 ? Ss 21:40 0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 21809 0.1 2.8 389244 112272 ? S 21:40 0:04 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 21853 0.0 0.0 4508 560 ? Ss 21:42 0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 21855 0.1 2.7 387196 109540 ? S 21:42 0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 21889 0.0 0.0 4508 708 ? Ss 21:43 0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 21891 0.1 2.7 387196 109756 ? S 21:43 0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 21930 0.0 0.0 4508 556 ? Ss 21:44 0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 21932 0.1 2.7 387196 109400 ? S 21:44 0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 21961 0.0 0.0 4508 636 ? Ss 21:45 0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 21963 0.1 2.7 387200 109760 ? S 21:45 0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 21997 0.0 0.0 4508 628 ? Ss 21:46 0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 22000 0.1 2.7 387200 110252 ? S 21:46 0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 22036 0.0 0.0 4508 556 ? Ss 21:47 0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 22038 0.1 2.7 387196 109752 ? S 21:47 0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 22079 0.0 0.0 4508 724 ? Ss 21:48 0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 22081 0.1 2.6 381052 103428 ? S 21:48 0:02 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 22126 0.0 0.0 4508 592 ? Ss 21:49 0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 22128 0.1 2.7 387196 109664 ? S 21:49 0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 22235 0.0 0.0 4508 704 ? Ss 21:51 0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 22236 0.1 2.7 385148 108676 ? S 21:51 0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 22533 0.0 0.0 4508 612 ? Ss 21:55 0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 22535 0.1 2.7 385148 107840 ? S 21:55 0:02 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 22700 0.0 0.0 4508 640 ? Ss 21:57 0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 22702 0.1 2.6 383100 105844 ? S 21:57 0:02 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 22782 0.0 0.0 4508 640 ? Ss 21:58 0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

orocomm+ 21755  0.0  0.0   4508   612 ?        Ss   21:39   0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 21759  0.1  2.8 389248 111772 ?       S    21:39   0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 21807  0.0  0.0   4508   608 ?        Ss   21:40   0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 21809  0.1  2.8 389244 112272 ?       S    21:40   0:04 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 21853  0.0  0.0   4508   560 ?        Ss   21:42   0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 21855  0.1  2.7 387196 109540 ?       S    21:42   0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 21889  0.0  0.0   4508   708 ?        Ss   21:43   0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 21891  0.1  2.7 387196 109756 ?       S    21:43   0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 21930  0.0  0.0   4508   556 ?        Ss   21:44   0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 21932  0.1  2.7 387196 109400 ?       S    21:44   0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 21961  0.0  0.0   4508   636 ?        Ss   21:45   0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 21963  0.1  2.7 387200 109760 ?       S    21:45   0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 21997  0.0  0.0   4508   628 ?        Ss   21:46   0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 22000  0.1  2.7 387200 110252 ?       S    21:46   0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 22036  0.0  0.0   4508   556 ?        Ss   21:47   0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 22038  0.1  2.7 387196 109752 ?       S    21:47   0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 22079  0.0  0.0   4508   724 ?        Ss   21:48   0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 22081  0.1  2.6 381052 103428 ?       S    21:48   0:02 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 22126  0.0  0.0   4508   592 ?        Ss   21:49   0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 22128  0.1  2.7 387196 109664 ?       S    21:49   0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 22235  0.0  0.0   4508   704 ?        Ss   21:51   0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 22236  0.1  2.7 385148 108676 ?       S    21:51   0:03 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 22533  0.0  0.0   4508   612 ?        Ss   21:55   0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 22535  0.1  2.7 385148 107840 ?       S    21:55   0:02 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 22700  0.0  0.0   4508   640 ?        Ss   21:57   0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null orocomm+ 22702  0.1  2.6 383100 105844 ?       S    21:57   0:02 /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod orocomm+ 22782  0.0  0.0   4508   640 ?        Ss   21:58   0:00 /bin/sh -c /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null

my crontab contains

Shell

# m h dom mon dow command */1 * * * * /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:cron --env=prod > /dev/null */1 * * * * /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null

1 2 3

# m h  dom mon dow   command */1 * * * * /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:cron --env=prod > /dev/null */1 * * * * /usr/bin/php /srv/orocommerce/orocommerce-application/app/console oro:message-queue:consume --env=prod > /dev/null

What is the problem please ?

Topic

[yannick.simonParticipant]

sorry, i just realized that i should not add a cron line for this oro:message-queue:consume

…

Should i put this command into a supervisord daemon ?

[wgozdzickiParticipant]

Hello Simon,
On production environment you should use Supervisord because it relaunch consumer every time when it dies. On development environment it’s convenient to use it. Also when you need more then one consumer you need Supervisord. You can find information about that here: https://www.orocrm.com/documentation/current/book/installation#activating-background-tasks and https://github.com/orocrm/platform/tree/master/src/Oro/Bundle/MessageQueueBundle#supervisord

Best regards

[yannick.simonParticipant]

thank you wgozdzicki

[saqibbhat96Participant]

Hello Simon, could you please share how to achieve supervisord configuration for message queuing. that would be helpful….
thanks

[yannick.simonParticipant]

Ok, no problem

my server is “ubuntu 16.04”

i have a directory /srv/orocommerce/orocommerce-application/
owned by orocommerce:orocommerce

installation of supervisord

Shell

apt-get install supervisor cd /etc/supervisord/conf.d/

1 2	apt-get install supervisor cd /etc/supervisord/conf.d/

i created this conf into /etc/supervisord/conf.d/orocommerce.conf

Shell

[program:oro_message_consumer] command=/srv/orocommerce/orocommerce-application/app/console --env=prod --no-debug oro:message-queue:consume process_name=%(program_name)s_%(process_num)02d numprocs=4 autostart=true autorestart=true startsecs=0 user=orocommerce redirect_stderr=true

1 2 3 4 5 6 7 8 9

[program:oro_message_consumer] command=/srv/orocommerce/orocommerce-application/app/console --env=prod --no-debug oro:message-queue:consume process_name=%(program_name)s_%(process_num)02d numprocs=4 autostart=true autorestart=true startsecs=0 user=orocommerce redirect_stderr=true

then, when i restart supervisord, i can see 4 processes

Shell

# /etc/init.d/supervisor restart ps aux|grep "php" orocomm+ 23200 0.1 3.2 391584 127516 ? S avril25 0:49 php /srv/orocommerce/orocommerce-application/app/console --env=prod --no-debug oro:message-queue:consume orocomm+ 23220 0.1 3.4 397792 135032 ? S avril25 0:46 php /srv/orocommerce/orocommerce-application/app/console --env=prod --no-debug oro:message-queue:consume orocomm+ 23236 0.1 3.2 393632 129732 ? S avril25 0:46 php /srv/orocommerce/orocommerce-application/app/console --env=prod --no-debug oro:message-queue:consume orocomm+ 23252 0.1 3.2 391584 127496 ? S avril25 0:47 php /srv/orocommerce/orocommerce-application/app/console --env=prod --no-debug oro:message-queue:consume

1 2 3 4 5 6 7

# /etc/init.d/supervisor restart ps aux|grep "php"   orocomm+ 23200  0.1  3.2 391584 127516 ?       S    avril25   0:49 php /srv/orocommerce/orocommerce-application/app/console --env=prod --no-debug oro:message-queue:consume orocomm+ 23220  0.1  3.4 397792 135032 ?       S    avril25   0:46 php /srv/orocommerce/orocommerce-application/app/console --env=prod --no-debug oro:message-queue:consume orocomm+ 23236  0.1  3.2 393632 129732 ?       S    avril25   0:46 php /srv/orocommerce/orocommerce-application/app/console --env=prod --no-debug oro:message-queue:consume orocomm+ 23252  0.1  3.2 391584 127496 ?       S    avril25   0:47 php /srv/orocommerce/orocommerce-application/app/console --env=prod --no-debug oro:message-queue:consume

[saqibbhat96Participant]

Hello Simon, thanks a lot for sharing the configuration and now it works, but my only concern is im able to see only one process

PHP

ps aux|grep "php" saaqib 5076 0.3 4.1 278616 167956 pts/1 T 09:00 0:12 /opt/bitnami/php/bin/php.bin app/console oro:message-queue:consume --env=prod saaqib 12307 0.0 0.0 12728 2168 pts/1 S+ 09:54 0:00 grep php saaqib 12308 0.0 0.0 5808 716 ? R 09:54 0:00 /usr/bin/env php /opt/bitnami/apps/b2b/htdocs/app/console --env=prod --no-debug oro:message-queue:consu me

1 2 3 4 5

ps aux|grep "php" saaqib    5076  0.3  4.1 278616 167956 pts/1   T    09:00   0:12 /opt/bitnami/php/bin/php.bin app/console oro:message-queue:consume --env=prod saaqib   12307  0.0  0.0  12728  2168 pts/1    S+   09:54   0:00 grep php saaqib   12308  0.0  0.0   5808   716 ?        R    09:54   0:00 /usr/bin/env php /opt/bitnami/apps/b2b/htdocs/app/console --env=prod --no-debug oro:message-queue:consu me

And regarding the cron jobs, you think this would be good to go cron jobs setup

And yeah i more important question. do i have to run supervisor as root?

Could you please see the matter.
Thanks

[yannick.simonParticipant]

saquibbhat96

i copied and paste the conf file from
https://github.com/orocrm/platform/tree/master/src/Oro/Bundle/MessageQueueBundle#supervisord

where the “numprocs=4” parameters seems to indicate that it must be run 4 times
and the “user=orocommerce” is the parameter i chose to run orocommerce on my server

i think only one oro:oro:message-queue:consume is ok,
but, i did not want to change anything from the config given on
https://github.com/orocrm/platform/tree/master/src/Oro/Bundle/MessageQueueBundle#supervisord

finally, you must have :
1 line in your crontab in order to run app/console oro:cron each minute
1 supervisor conf file in order to run at least 1 app/console oro:message-queue:consume continually (it will restart if crashed or finish, thanks to supervisor)

I think that you should not run any command as root for orocommerce,
when, i installed supervisor, and created the supervisor conf file : i was logged as root, only for that.

[saqibbhat96Participant]

Hello Simon,
the supervisor is working fine thanks to you.
But regarding the setup of Cron jobs, i tried to write in

PHP

sudo crontab -e

1	sudo crontab -e

Adding a line

PHP

*/1 * * * * /usr/bin/php/bin/php /opt/bitnami/apps/b2b/htdocs/app/console oro:cron --env=prod > /dev/null

1	*/1 * * * * /usr/bin/php/bin/php /opt/bitnami/apps/b2b/htdocs/app/console oro:cron --env=prod > /dev/null

But when i save the file, and check the

PHP

crontab -l

1	crontab -l

output “no crontab set for this user”, and the line goes missing from the crontab -e

please assit, is there any other way of setting up cron jobs?

[yannick.simonParticipant]

saqibbhat96

the reason is that each user owns a specific crontab

when you enter

PHP

sudo crontab -e

1	sudo crontab -e

with this command you change root’s crontab

after that, you enter

PHP

crontab -l

1	crontab -l

with this command you list your current user’s crontab

my advice is to remove the line you entered with “sudo crontab -e”

and to add the same line with “crontab -e”
(without sudo)

[saqibbhat96Participant]

thanks Simon, cron is working fine.

Best Wishes

[Author]

Replies