Comments to Author:
===================

Both reviewers like the idea of using neural networks to
combine different input sources with uncertainty estimation
in the context of driver trajectory prediction. Both
reviewers find the paper novel enough to be of interest to
the community.

The main concerns of the two reviewers are:
1) A lack of details on the network architecture. As a
result, replicating the paper will be hard. 
2) The paper seems to oversell the performance of the
proposed approach. In addition, one of the reviewers
criticized a lack of comparisons with previous work.

I very much agree with the first concern. Not providing
details on the network architecture and the optimization
algorithm used to train the networks (including parameters
such as learning rate) will prevent others from replicating
the paper. It might even prevent other researchers from
pursuing a similar path as it will be unclear whether or
when their work improves upon the paper under review or
simply replicates it (especially as the data used in the
paper under review does not seem to be available), which
will make such further work hard to publish. However, this
issue could be addressed in the camera ready version of the
paper by adding details on the network architectures, etc.

I agree with the concern that the paper seems to be overly
optimistic in terms of the performance of the proposed
approach. On the positive side, since the problem
considered in the paper is highly relevant for robotics,
the current performance could inspire other work on this
topic.
However, this would require that the paper is reproducible.
In my opinion, the paper provides an adequate comparison
with previous work in Sec. IV.B.2 when it compares with
RNN-based approaches.

----------------------------------------
Comments on Video Attachment:

As both reviewers agree, the video nicely illustrates the
proposed approach and also provides qualitative examples.
One of the reviewers suggests adding an audio track, but I
feel that the video is clear already.



===================
First reviewer

The authors present an ego-vehicle path prediction method that utilizes different types of input sensors such as image, past trajectory and speed. They combine	data driven and a physics-based ones thanks to confidence estimator.
The structure of the data-driven method, which is the same as the confidence estimator, is based on different small NNs each of them estimates the output of each sensor's
output. 


Paper's strengths:
The paper is novel in terms of using both data-driven and a physics-based methods with confidence estimation. Also, as there are different input sources in a car, the way of
combining them is important since they have different scales and meanings. In this paper they solve this issue using NNs which is interesting.

Paper's weaknesses:
- It is not clear that the performance is really good or not as it is not compared with any other methods. It is essential to have comparisons in different ways. If there
are no similar work, first it should be mentioned and next, it will be good if it does some simulations of other solutions and try to compare this work with them by
changing some of the inputs or some of the situations in the problem.
- The paper lacks technical details. In terms of reproducibility, it is weak. The data, the physics-based algorithm and the details of the networks are missing. So, it would be a paper which won't be reimplemented again and compared with others.

Some additional comments:
- Page 3, paragraph 2, after "Gaussian mixture model " a"(GMM) should be added. [Done]
- Page 4, additional expert predictor should be specified and cited. [-]
- Page 4, in "confidence estimator training", If the confidence values will be meaningful outside the module, as it is mentioned in the introduction, the loss function could be important. But if it is just a comparison between different methods, it is not important.
- Fig 7,a, The colors in the figure aren't chosen well
- Fig 7,b, It seems that the estimators are doing the same relating to the time horizon. So, it is beneficial to show by experiments in which scenes odometry is better than variational.
- Fig 7, c, How do you explain the falling part in the underestimated curve?
- What is the structure of the mixture predictor?
- In different parts of the paper it is mentioned that "additional expert predictor provides more accurate predictions over shorter horizons" but it is not similar to the figures

===================
Second reviewer

This paper presents a prediction method for driver
trajectory prediction with uncertainty estimates. The paper
focuses on applications in urban environments, specifically
at intersections. The authors propose a hierarchical
network to solve the problem. There are several children
that process various sensor data. The output of these
children networks are used as inputs to a trajectory
predictor. An physics-based predictor network based on
wheel encoder data also provides trajectory estimates. In
addition, a separate network is trained to generate
confidence estimates. The outputs of the trajectory
predictor, physics-based network, and confidence estimates
are fed into a final mixture predictor to generate the
final trajectory predictors.

While the approach borrows idea from previous research
papers, the final application is novel and exciting. Having
uncertainty estimates associated with a generated
trajectory enables the output to be integrated into a more
sophisticated autonomous driving system. While the results
are presented as high performing, they are not ideal. The
average endpoint error of the trajectories over a 3.0s
horizon is over 3m, which would bring the vehicle into the
wrong lane. That being said, the performance is much better
over short horizons. It would be interesting to validate
the results of this network in a closed-loop environment to
get a better sense of the results.

The paper is overall well written and gives good intuition
for the decisions for the network in the paper. That being
said, the paper lacks more specifics on the network
architecture (number of layers, etc.). I have also noted
some small typos below:

page 3: "provides a warning if none of the candidates are
trustworthy" [Done]
page 4: "... we can use more than one expert predictor." [Done]
page 4: "... trained with different loss functions." [Done]