Low volatile GPU-Util but high GPU Memory Usage

[deipss]

When I run MLP.py ,GMF.py or ,NeuMF.py with my GPU (GTX 1080), the memory of which is fulled while the volatile GPU-Util are always around 20%. And the time-spending with GPU is slower than CPU (intel i5)。

I guess the problem are caused by

        # Training        
        hist = model.fit([np.array(user_input), np.array(item_input)], #input
                         np.array(labels), # labels 
                         batch_size=batch_size, epochs=1, verbose=0, shuffle=True)

in the codes of above, np.array(user_input), np.array(item_input),np.array(labels) is runing on the CPU but GPU.
Therefore, the useage of GPU-Util can not raise.

[jsmatte]

@deipss any suggestions on how to solve this issue? I came to the same realization and am trying to find a work around. Any help is much appreciated!

[edervishaj]

I have experienced the same slower training times with a GPU. Investigating the execution with py-spy, it seems that a big chunk of the training time is spent building user_input, item_input and labels.