Replies: 10 comments 12 replies
-
|
Dear All, Here is some more information. I attach the compile script I am using (thank you to Josh Harris for providing this script) along with the output produced when running the script. You can see the line "checking for MA57... no" which means MA57 is not present. I note, however, another line that says, "configure: Configuration of CoinHSL successful". Amongst other issues that may be present, it seems as if the HSL libraries are not being compiled into IPOPT despite the fact that the the flag "COMPILE_IPOPT_USE_HSL" at the top of the script is set to "true". I attach the script and the output of running the script. I am grateful if anyone can tell me why mumps and ma57 are not being compiled into IPOPT. |
Beta Was this translation helpful? Give feedback.
-
|
Ipopt's configure doesn't pick up your MUMPS installation because pkg_config is not available. This is required if you do not want to specify flags for compiling and linking against MUMPS manually (https://coin-or.github.io/Ipopt/INSTALL.html#SYSTEMPACKAGES_MAC).
One gets |
Beta Was this translation helpful? Give feedback.
-
|
And one more thing. Here is the output of "make test" for IPOPT. As it might be seen, the compilation is unable to locate "libhsl.dylib". This makes sense given that, for some reason, the HSL libraries are not being compiled into IPOPT (please see my prior output in the file compilation.txt with the line "checking for MA57... no". Thanks again for any help. — Anil Making all in src Running unitTests... Testing AMPL Solver Executable... EXIT: Library loading failure. *** The problem FAILED! Test program existed with nonzero status. EXIT: Library loading failure. ERROR OCCURRED DURING IPOPT OPTIMIZATION. Test program existed with nonzero status. Program received signal SIGABRT: Process abort signal. Backtrace for this error: Test program existed with nonzero status. ---- 8< ---- Start of test program output ---- 8< ---- EXIT: Library loading failure. #------------------------------------------- Sensitivity without bound checking#------------------------------------------- Internal abort has been called for the sensitivity calculations. #------------------------------------------- Sensitivity with bound checking#------------------------------------------- Internal abort has been called for the sensitivity calculations. Output of the test program does not contain 'EXIT: Optimal Solution Found'. ---- 8< ---- Start of test program output ---- 8< ---- EXIT: Library loading failure. Internal abort has been called for the sensitivity calculations. Output of the test program does not contain 'EXIT: Optimal Solution Found'. *** Solve for 0 variables, 0 feasible constraint, feasible bounds EXIT: Library loading failure. Test program existed with nonzero status. |
Beta Was this translation helpful? Give feedback.
-
|
I've made a few more changes but to no avail. It turns out that the name of the unpacked HSL directory was incorrect. I have now changed it so that, using the script in my previous message, the name is now "coinhsl" (that is, the file "coinhsl-2022.11.09.tar.gz untars into "coinhsl"). That now seems to be correct as the directory coinhsl exists in "ThirdParty-HSL". This directory coinhsl contains the MA57 source code along with its dependencies as obtained from Harwell. Thus, I believe the compilation should detect and compile into Ipopt the MA57 solver (well, at least I am hoping I am correct). Nevertheless, nothing has changed even with the aforementioned modification. I still cannot run "make test" successfully and I still receive the error that the HSL libraries cannot be found and that it is trying to find the dynamic library libhsl.dylib (same output as before). Anyway, just wanted to post another update in case it is at all helpful. |
Beta Was this translation helpful? Give feedback.
-
|
Here is yet some more information. The MA57 source code files definitely exist in the directory "ThirdParty-HSL/coinhsl/src". after running the configure script in "ThirdParty-HSL" I see that the lines below are all commented out. Should it not be the case that at least the following two lines should NOT be commented out: #am__append_7 = coinhsl/ma57/ma57d.f I am beginning to think that the configure script in ThirdParty-HSL is not properly detecting the MA57 source code even though it resides in the directory ThirdParty-HSL/coinhsl/src. Again, just more information in case anyone can help diagnosing the problem. Thanks! — Anil #am__append_1 = coinhsl/mc19/mc19d.f |
Beta Was this translation helpful? Give feedback.
-
|
Don't know why the MA57 source ( |
Beta Was this translation helpful? Give feedback.
-
|
Dear All, More information. I think the problem with my compilation is that neither MUMPS nor MA57 is being compiled into IPOPT. Thus, while I am generating both libcoinmumps.a and libcoinhsl.a libraries (which are being saved in /usr/local/lib), neither of these libraries actually contains MUMPS and MA57, respectively. I know that might sound strange, but the error I am getting for MUMPS is that it is not a valid linear solver option, while the error I am getting for MA57 is that the dynamic library is not being found. I note, however, that the IPOPT Mex file compile script is being linked to libcoinmumps.a and libcoinhsl.a via the linker options -lcoinmumps and -lcoinhsl in the file "CompileIpoptMexLib.m". This last point seems to imply that these libraries are "empty" (that is, they do not contain compiled libraries of either linear solver). With that explanation given, I have no idea why what I am saying above would be the case as I set up everything exactly as the instructions are provided on the IPOPT GitHub page. Again, any help is greatly appreciated. Once I get this whole thing figured out, perhaps I have some kind of a chance to create yet another Mex file that links to the MATLAB MA57 library. Thanks! — Anil |
Beta Was this translation helpful? Give feedback.
-
If the Ipopt code has been compiled without being aware that it will be linked against HSL or MUMPS, then just linking these libs into the Ipopt/MEX library later won't change anything. |
Beta Was this translation helpful? Give feedback.
-
|
Dear Stefan et al, Thank you so much for your help. I have good news and bad news. First, the good news. I installed pkg-config and moved the MA57 source code to the directory coinhsl/ma57. After this revision, the configuration now find the MA57 source code. Now the bad news. I am getting the error during the compilation of the HSL libraries: configure: error: Required file common/deps.f not found In looking in the coinhsl directory, I see no subdirectory "common". I have a (very old) version of HSL (from 2014) which does indeed have the directory "common" and contains the file deps.f. (interestingly, this older version of the HSL libraries seems to contain all of the linear solvers). n any case, I am not sure if I need to request a "complete" package from Harwell or if these files are located elsewhere in these later releases. Thanks for your help. — Anil |
Beta Was this translation helpful? Give feedback.
-
|
I cannot look into current releases. Just use the 2014 one then. Not much has changed since then. |
Beta Was this translation helpful? Give feedback.
-
|
I now used my older version of the HSL source code (from 2014). I get a bunch of errors because the code is not completely compatible with the 2018 and beyond compilers, but I think they are just warnings. In any case, when I am done with the compilation and I run "make test" in the Ipopt/build directory, I now receive the following error: clang: error: unsupported option '-fopenmp' I get that same error when using gcc/g++. I do not understand why the flag "-fopenmp" is not supported. I also note that neither clang nor clang++ appears anywhere in the Makefile in the Ipopt/build directory nor in the Ipopt/build/test directory. The compilers used by the Makefile are "CC = gcc" and "CXX = g++". Thus, I have no idea why, then, I am receiving a "clang" error even when I am not using clang and it is nowhere in the makefile. Again, thanks for the help. — Anil |
Beta Was this translation helpful? Give feedback.
-
|
On clang for macOS on arm64, |
Beta Was this translation helpful? Give feedback.
-
|
I am using the flags -Xpreprocessor -fopenmp. I also now switched to the llvm versions of clang and clang++. After doing all of that, I am able to build IPOPT with the 2014 version of HSL, but the make test for IPOPT does not run correctly. Make test compiles, but it aborts every example after zero iterations. |
Beta Was this translation helpful? Give feedback.
-
|
After compiling with the LLVM versions of clang and clang++, I am able to successfully complete the compilation of IPOPT. When I test the build of IPOPT, however, make test fails. The output from "make test" in the Ipopt/build directory is shown below. I have no idea what is going on. -- Anil Making all in src Running unitTests... Testing AMPL Solver Executable... *** Solve for 0 variables, 0 feasible constraint, feasible bounds This program contains Ipopt, a library for large-scale nonlinear optimization. This is Ipopt version 3.14.16, running with linear solver ma27. Number of nonzeros in equality constraint Jacobian...: 0 Total number of variables............................: 0 iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls Number of Iterations....: 0 Objective...............: 0.0000000000000000e+00 0.0000000000000000e+00 Number of objective function evaluations = 1 EXIT: Optimal Solution Found. The problem solved in 0 iterations! The final value of the objective function is 0. *** Solve for 0 variables, 1 feasible constraint, feasible bounds Number of nonzeros in equality constraint Jacobian...: 0 Total number of variables............................: 0 iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls Number of Iterations....: 0 Objective...............: 0.0000000000000000e+00 0.0000000000000000e+00 Number of objective function evaluations = 1 EXIT: Optimal Solution Found. The problem solved in 0 iterations! The final value of the objective function is 0. *** Solve for 5 variables, 1 feasible constraint, feasible bounds Number of nonzeros in equality constraint Jacobian...: 0 Total number of variables............................: 0 iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls Number of Iterations....: 0 Objective...............: 0.0000000000000000e+00 0.0000000000000000e+00 Number of objective function evaluations = 1 EXIT: Optimal Solution Found. The problem solved in 0 iterations! The final value of the objective function is 0. *** Solve for 0 variables, 1 infeasible constraint, feasible bounds Test program existed with nonzero status. |
Beta Was this translation helpful? Give feedback.
-
|
More information. When I switch back to the original clang and clang++ (which are located in /usr/bin/clang and /usr/bin/clang++) the compilation of IPOPT is successful but when I run "make test" in the Ipopt/build directory I receive the error "clang: error: unsupported option '-fopenmp'. On the other hand, when I use the clang and clang++ in the directories "/opt/homebrew/Cellar/llvm/18.1.7/bin/clang" and "/opt/homebrew/Cellar/llvm/18.1.7/bin/clang++", respectively, I do not receive the "-fopenmp" error in make test. Instead, I receive the output I just posted where the tests are run but they fail (see my post from a few minutes before this post). Again, I am wholly confused. I am using "-Xpreprocessor -fopenmp" in both my CFLAGS and CPPFLAGS in both cases. I do not know what else to report with regard to my compilation or any other information that might be useful. Thanks! — Anil |
Beta Was this translation helpful? Give feedback.
-
|
Adding I would suggest just disabling OpenMP. Configure with |
Beta Was this translation helpful? Give feedback.
-
|
I tried disabling openmp in the IPOPT configure command. I still get the same result as before. Make test fails the same way as the output I showed previously. |
Beta Was this translation helpful? Give feedback.
-
|
Please disable it in the configure command for ThirdParty-HSL and make sure you don't enable it for ThirdParty-Mumps. Ipopt doesn't use OpenMP. |
Beta Was this translation helpful? Give feedback.
-
|
I disabled openmp everywhere (Mumps, HSL, Ipopt), etc and removed "-fopenmp" in the configure commands for Mumps, HSL, and Ipopt. In all configure commands (Ipopt, Mumps, and HSL) I included "--disable-openmp". I also switched back to the original versions of clang and clang++ (/usr/bin/clang and /usr/bin/clang++) for my compilers (rather than using /opt/homebrew/Cellar/llvm/18.1.7/bin/clang and /opt/homebrew/Cellar/llvm/18.1.7/bin/clang++ or /opt/homebrew/opt/llvm/bin/clang and /opt/homebrew/opt/llvm/bin/clang++. Basically, I tried to revert back to my original options but with openMP excluded. Now I am able to run make test and the output is below. I note, however, that when I compile hs_071 I still get the error that the compilation is looking for the HSL dynamic library (see the next output below). I have no idea why the compilation wants to load the dynamic HSL library when I have used --with-hsl as an option in the IPOPT configure command. Does using --with-hsl set the compiled Ipopt library to include the HSL solver, or is some other option to be used for making sure that HSL is compiled into the Mex file. Next, I do not know how to test whether Mumps has been compiled into the Ipopt library because I do not see any lines in the examples that set the linear solver. I did, however, compile the Ipopt Mex interface and tried both MA57 and Mumps. For MA57 I get the same error about wanting to load the HSL library. For Mumps I repeated receive the error that "mumps" is not a valid option for a linear solver (see that last output below). Thanks again for the help. — Anil |
Beta Was this translation helpful? Give feedback.
-
|
Would need to see the output of the whole procedure.
|
Beta Was this translation helpful? Give feedback.
-
|
OK, I feel like somewhat (only somewhat) of an idiot regarding the compilation of an IPOPT Mex file for Apple Silicon. I had a discussion last week with Josh Harris (who graciously sent me his compile script). It turns out that the directory pkgconfig within /usr/local/lib was not being overwritten each time I tried the compilation (once I got everything set up correctly). Inside that directory are three files: coinhsl.pc, coinmumps.pc, and ipopt.pc. These three files were not being overwritten. As a result, whenever I tried to run the Mex file that I had compiled, the execution pointed to a previous compilation (which I had done a while back). Consequently, nothing worked correctly. After deleting the contents of /usr/local/lib (I actually just deleted everything) and recompiling, I was able to successfully compile an IPOPT Mex file for Apple Silicon that works with all of the HSL linear solvers. What an adventure! I will post a step-by-step process to make everything formal so that anyone can try and produce their own IPOPT Mex file for use with Mumps and the HSL solvers (assuming one has the HSL source code). Please give me a few days and I will get back to everyone with what I hope to be a repeatable process. Thanks! — Anil |
Beta Was this translation helpful? Give feedback.
-
|
Dear Anil, any update about the procedure? I think that it would be something valuable for many IPOPT/AppleSilicon users.... All the best, |
Beta Was this translation helpful? Give feedback.
-
Dear All
I seem to never be able to compile an IPOPT Mex file on Mac.
I am currently trying to compile a Mex file on Apple Silicon using the MA57 and MUMPS source code. I am able to compile IPOPT and then use Enrico Bertolazzi's Mex compilation script to compile a Mex file successfully. I have used the options "--enable-static=yes --enable-shared=no” for both MUMPS and MA57. After creating the Mex file, I receive the following two run time errors. First, when setting the linear solve to "ma57" I receive the error as follows:
Exception of type: DYNAMIC_LIBRARY_FAILURE in file "/Users/anilvrao/Documents/MATLAB/harrisIpopt/Ipopt/src/Common/IpLibraryLoader.cpp" at line 86:
Exception message: dlopen(libhsl.dylib, 0x0002): tried: 'libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibhsl.dylib' (no such file), '/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/Applications/MATLAB_R2024a.app/bin/maca64/libhsl.dylib' (no such file), '/Applications/MATLAB_R2024a.app/bin/maca64/./libhsl.dylib' (no such file), '/Applications/MATLAB_R2024a.app/bin/maca64/../../sys/os/maca64/libhsl.dylib' (no such file), '/Applications/MATLAB_R2024a.app/bin/maca64/../../runtime/maca64/libhsl.dylib' (no such file), '/Applications/MATLAB_R2024a.app/bin/maca64/../../sys/java/jre/maca64/jre/lib/server/libhsl.dylib' (no such file), '/Applications/MATLAB_R2024a.app/bin/maca64/../../sys/java/jre/maca64/jre/lib/libhsl.dylib' (no such file), '/usr/lib/libhsl.dylib' (no such file, not in dyld cache), 'libhsl.dylib' (no such file)
Next, when I try to use "mumps" as the linear solver I receive the error
Error using ipopt
*** Error using Ipopt Matlab interface: ***
Invalid value for IPOPT option "linear_solver"
I have no idea what is going on. How can it be that mumps is not an option in ipppt? Also, why is the Mex file looking for a dynamic library to load when I am using the Harwell source code and I am enabling static and disabling shared.
I am so confused. Any help is appreciated.
Regards,
Anil
Beta Was this translation helpful? Give feedback.
All reactions