advertisement


Keras - I can't see the wood for the trees

Darth Vader

Darth Vader

From the Dark Side
Another of my long shots as I believe that there are some knowledgeable folks on pfm.

I have installed keras and tried to verify it is working. However the command python <file path> test.py fails to complete with an 'unable to load module keras at line x'. Whether I use the full path name or execute from within the keras directory I get the same result.

Now if I invoke python (version 3.7) from within the keras directory and paste the code from test.py it works. So I know keras works and I can monitor the GPU stats to see how hard the training is working my GTX 1080.

So it has to be something to do with the environment variables as python doesn't seem to know the path to keras.

Anyone have a clue? I can carry on working by pasting code into python whilst sitting in the keras directory but its not very sophisticated is it? The answer is probably right under my nose............

BTW I don't get this problem with tensorflow - at last!

Many thanks in advance,

DV
 
Why are you asking this on a hifi forum when Keras has its own support forum.
 
I don't know about python as my brain (sic) has limited at 'C'. However if it is a path problem maybe its an package install process that's missed getting this right?
 
thebiglebowski said:
Why are you asking this on a hifi forum when Keras has its own support forum.
Click to expand...
Because I think it is a Linux or maybe python path issue and others on pfm may have met this sort of thing before. As I wrote above the keras code works and runs on my GPU fast. So its an issue that python has with running the same code but from a file rather than directly. Also the code fails if I invoke python from my home directory so points again to a Linux path/environment issue.

Cheers,

DV
 
Jim Audiomisc said:
I don't know about python as my brain (sic) has limited at 'C'. However if it is a path problem maybe its an package install process that's missed getting this right?
Click to expand...
I think you are spot on. I have had a lot of issues due to the various upgrades of these multiple 3rd party packages and have had to re-establish paths/variables etc that became broken.

Thing is I can carry on working but I would like a more elegant and pro solution although its really not that important for my 'training'. I am amazed at the power available today and for free! I've recently been playing with pandas - the reference guide alone exceeds 3000 pages! However in a few short lines of code I could manipulate a dataset getting on for 50K records with 37 fields and it was simpler (well for me) than the Excel equivalent. Once I have my ML up to a reasonable standard pandas with numpy looks exciting and may be better than Excel.

I still think I am missing something obvious but with all these packages its like a needle in a haystack or in my case not seeing the wood for the trees. Oh and don't mention random forests.

Cheers,

DV
 
Last edited:
Darth Vader said:
I think you are spot on. I have had a lot of issues due to the various upgrades of these multiple 3rd party packages and have had to re-establish paths/variables etc that became broken.

Thing is I can carry on working but I would like a more elegant and pro solution ...

I still think I am missing something obvious but with all these packages its like a needle in a haystack or in my case not seeing the wood for the trees. Oh and don't mention random forests.

Cheers,

DV
Click to expand...

I've just spent about three hours trying to work out why my usual audio recording program wouldn't run on a new Linux Mint install... only to finally twig it was a side-effect of 'xterm' not being installed and my getting meaningless or silent failures.

Much time looking in all thr wrong places...
 
darrenyeats said:
Assuming you're invoking python as 'python' (and not './python' in a Keras bin folder or something and therefore invoking a different python) then I guess the python library path has something to do with it (maybe it includes '.'). https://stackoverflow.com/questions/135035/python-library-path
Click to expand...
Thanks for this. No I haven't solved the problem but now I know it is both a global O/S and python environment issue. I think the mess is caused because Unbuntu comes with python 2.x installed as its needed by the kernel. However I am using python 3.7. The installation of other modules interact with both versions of python and pip installation is different from the bundled package manager. For example I have updated pip to the latest v19 and pip -V shows this. However the package manager shows only version 8 installed. Then when invoking installation of 3rd party modules I sometimes get a message saying that it is using version 8 and I should upgrade to version 19!

Its like looking for a needle in a haystack!

So I have a simple work around that executes python scripts directly in python. I use the sys.path.append (from the link above) to point to the path for keras stuff and then bring the python script in with:- from <dir>.<file> import *.

Cheers,

DV
 
If you're doing DL I'd look at Anaconda, which is Python bundled with data science libs and Jupyter Notebook. That's what I used for the DL foundation course with Udacity.
 
darrenyeats said:
If you're doing DL I'd look at Anaconda, which is Python bundled with data science libs and Jupyter Notebook. That's what I used for the DL foundation course with Udacity.
Click to expand...
Yes thats what I have installed Anaconda3. I probably should have uninstalled all the earlier ML stuff that I used to have-a-play before adding the new stuff to teach myself however apart from these niggles 'it' = keras/tensorflow/numpy/ pandas/python/CUDA etc etc all seem to be working together.

Its just that as an ex pro the work around doesn't come across as 'elegant' if you know what I mean.

Cheers,

DV
 
I was hoping not having to go and create environments as I am lazy and just want to get on working with ML/DL. However the command line that you provided gives exactly the same result as:-

conda env list
# conda environments:
#
base * /home/darth/anaconda3

So no root entry.

I'll do some more reading as I was toying with the idea of installing a new Linux O/S on a spare disk partition so a new environment may be less hassle anyway

Thanks for the pointers,.

DV
 
You must log in or register to reply here.


advertisement


Back
Top