vinyl-cache/vinyl-cache

Fork 8

[4.1/master] vmodtool parallell make dependency problem #1866

New issue

Closed

opened 2016-03-04 16:20:49 +01:00 by lkarsten · 10 comments

lkarsten commented

2016-03-04 16:20:49 +01:00

(Migrated from github.com)

From time to time the build jobs in jenkins fail, usually recovering after a manual rebuild.

I think this is because vmodtool isn't done writing vcc_if.[ch], but I haven't verified this.

Below is 4.1 but should be identical in master.

make[3]: Leaving directory `/home/jenkins/workspace/varnish-4.1-build-el6-x86_64/varnish-4.1.2-beta2/lib/libvmod_debug'
Making all in libvmod_std
make[3]: Entering directory `/home/jenkins/workspace/varnish-4.1-build-el6-x86_64/varnish-4.1.2-beta2/lib/libvmod_std'
python2.6 ../../lib/libvcc/vmodtool.py --strict ../../lib/libvmod_std/vmod.vcc
python2.6 ../../lib/libvcc/vmodtool.py --strict ../../lib/libvmod_std/vmod.vcc
  CC     vmod_std.lo
  CC     vmod_std_fileread.lo
  CC     vmod_std_conversions.lo
  CC     vmod_std_querysort.lo
cc1: warnings being treated as errors
vmod_std_conversions.c:49: error: no previous prototype for 'vmod_duration'
vmod_std_conversions.c:103: error: no previous prototype for 'vmod_integer'
vmod_std_conversions.c:130: error: no previous prototype for 'vmod_ip'
vmod_std_conversions.c:173: error: no previous prototype for 'vmod_real'
vmod_std_conversions.c:191: error: no previous prototype for 'vmod_real2integer'
vmod_std_conversions.c:204: error: no previous prototype for 'vmod_real2time'
vmod_std_conversions.c:215: error: no previous prototype for 'vmod_time2integer'
vmod_std_conversions.c:228: error: no previous prototype for 'vmod_time2real'
vmod_std_conversions.c:239: error: no previous prototype for 'vmod_time'
make[3]: *** [vmod_std_conversions.lo] Error 1
make[3]: *** Waiting for unfinished jobs....
cc1: warnings being treated as errors
vmod_std_fileread.c:85: error: no previous prototype for 'vmod_fileread'
make[3]: *** [vmod_std_fileread.lo] Error 1
cc1: warnings being treated as errors
vmod_std_querysort.c:53: error: no previous prototype for 'vmod_querysort'
make[3]: *** [vmod_std_querysort.lo] Error 1
make[3]: Leaving directory `/home/jenkins/workspace/varnish-4.1-build-el6-x86_64/varnish-4.1.2-beta2/lib/libvmod_std'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/jenkins/workspace/varnish-4.1-build-el6-x86_64/varnish-4.1.2-beta2/lib'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/jenkins/workspace/varnish-4.1-build-el6-x86_64/varnish-4.1.2-beta2'
make: *** [all] Error 2
Build step 'Execute shell' marked build as failure
Notifying upstream projects of job completion
Finished: FAILURE

Job link: https://jenkins.varnish-software.com/view/VC-4.1/job/varnish-4.1-build-el6-x86_64/

From time to time the build jobs in jenkins fail, usually recovering after a manual rebuild. I think this is because vmodtool isn't done writing vcc_if.[ch], but I haven't verified this. Below is 4.1 but should be identical in master. ``` make[3]: Leaving directory `/home/jenkins/workspace/varnish-4.1-build-el6-x86_64/varnish-4.1.2-beta2/lib/libvmod_debug' Making all in libvmod_std make[3]: Entering directory `/home/jenkins/workspace/varnish-4.1-build-el6-x86_64/varnish-4.1.2-beta2/lib/libvmod_std' python2.6 ../../lib/libvcc/vmodtool.py --strict ../../lib/libvmod_std/vmod.vcc python2.6 ../../lib/libvcc/vmodtool.py --strict ../../lib/libvmod_std/vmod.vcc CC vmod_std.lo CC vmod_std_fileread.lo CC vmod_std_conversions.lo CC vmod_std_querysort.lo cc1: warnings being treated as errors vmod_std_conversions.c:49: error: no previous prototype for 'vmod_duration' vmod_std_conversions.c:103: error: no previous prototype for 'vmod_integer' vmod_std_conversions.c:130: error: no previous prototype for 'vmod_ip' vmod_std_conversions.c:173: error: no previous prototype for 'vmod_real' vmod_std_conversions.c:191: error: no previous prototype for 'vmod_real2integer' vmod_std_conversions.c:204: error: no previous prototype for 'vmod_real2time' vmod_std_conversions.c:215: error: no previous prototype for 'vmod_time2integer' vmod_std_conversions.c:228: error: no previous prototype for 'vmod_time2real' vmod_std_conversions.c:239: error: no previous prototype for 'vmod_time' make[3]: *** [vmod_std_conversions.lo] Error 1 make[3]: *** Waiting for unfinished jobs.... cc1: warnings being treated as errors vmod_std_fileread.c:85: error: no previous prototype for 'vmod_fileread' make[3]: *** [vmod_std_fileread.lo] Error 1 cc1: warnings being treated as errors vmod_std_querysort.c:53: error: no previous prototype for 'vmod_querysort' make[3]: *** [vmod_std_querysort.lo] Error 1 make[3]: Leaving directory `/home/jenkins/workspace/varnish-4.1-build-el6-x86_64/varnish-4.1.2-beta2/lib/libvmod_std' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/home/jenkins/workspace/varnish-4.1-build-el6-x86_64/varnish-4.1.2-beta2/lib' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/jenkins/workspace/varnish-4.1-build-el6-x86_64/varnish-4.1.2-beta2' make: *** [all] Error 2 Build step 'Execute shell' marked build as failure Notifying upstream projects of job completion Finished: FAILURE ``` Job link: https://jenkins.varnish-software.com/view/VC-4.1/job/varnish-4.1-build-el6-x86_64/

dridi commented

2016-03-07 13:33:12 +01:00

(Migrated from github.com)

I ran into the same issue in vmod-example, so I first declared all vmodtool.py output files to make:

$(VCC_FILES) $(DOC_FILES): vmod_example.vcc
        @echo vmodtool: @VMODTOOL@ $<
        @@VMODTOOL@ $<

It could still run up to 4 times (confirmed with make -j 32) but I would always get proper output. After a comment from @gquintard I ended up using a temp file to fix that too.

See https://github.com/varnish/libvmod-example/pull/30#issuecomment-184592855

I ran into the same issue in vmod-example, so I first declared all `vmodtool.py` output files to `make`: ``` make $(VCC_FILES) $(DOC_FILES): vmod_example.vcc @echo vmodtool: @VMODTOOL@ $< @@VMODTOOL@ $< ``` It could still run up to 4 times (confirmed with `make -j 32`) but I would always get proper output. After a comment from @gquintard I ended up using a temp file to fix that too. See https://github.com/varnish/libvmod-example/pull/30#issuecomment-184592855

nigoroll commented

2016-03-07 13:37:59 +01:00

(Migrated from github.com)

Before I committed 82d2522712892b6edffdc4d805ce7400c52d1f34 I found no other way than to set the vmodtool build target to one file and create a dependency of all other files to that file

dridi commented

2016-03-07 14:31:17 +01:00

(Migrated from github.com)

@nigoroll where's the sha1 pointing to?

nigoroll commented

2016-03-07 14:43:44 +01:00

(Migrated from github.com)

github.com/varnish/libvmod-example@82d2522712

https://github.com/varnish/libvmod-example/commit/82d2522712892b6edffdc4d805ce7400c52d1f34

dridi commented

2016-03-07 14:48:56 +01:00

(Migrated from github.com)

Ah, I've actually seen that but wasn't fond of the idea of having output from a same target being dependencies of each other. But I changed it only because I had introduced variables (VCC_FILES and DOC_FILES) for those files that get referenced in several places.

Ah, I've actually seen that but wasn't fond of the idea of having output from a same target being dependencies of each other. But I changed it only because I had introduced variables (`VCC_FILES` and `DOC_FILES`) for those files that get referenced in several places.

nigoroll commented

2016-03-07 15:01:52 +01:00

(Migrated from github.com)

On 07/03/16 14:48, Dridi Boukelmoune wrote:

but wasn't fond of the idea of having output from a same target being
dependencies of each other.

I don't like it either, but this is just how make works.

On 07/03/16 14:48, Dridi Boukelmoune wrote: > but wasn't fond of the idea of having output from a same target being > dependencies of each other. I don't like it either, but this is just how make works.

dridi commented

2016-03-09 15:51:11 +01:00

(Migrated from github.com)

@lkarsten please let me know whether this issue still happens when building the master branch. I'll take care of back-porting the fix to maintenance branches and close this issue.

nigoroll commented

2016-06-13 23:19:23 +02:00

(Migrated from github.com)

@Dridi we could close this one if you're done with backporting

lkarsten commented

2016-07-04 14:28:05 +02:00

(Migrated from github.com)

This was fixed in vmodtool.py in 6693cfc538. Closing.

This was fixed in vmodtool.py in 6693cfc538883dd9f5e99c186aaf78c9829e6fa0. Closing.

hermunn commented

2016-07-11 10:58:16 +02:00

(Migrated from github.com)

This was backported in 0b1ff5170e, removing tag.

This was backported in 0b1ff5170ed56484db73ee3a6396830dae01ebf5, removing tag.

This discussion has been locked. Commenting is limited to contributors.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

vinyl-cache/vinyl-cache#1866

No description provided.

Rows
Columns