Errors when some static libs are combined into one single dynamic lib.

JoeCFD · 22 Dec 2023, 16:56

In one of my apps, I need to combine a few C static libs into one single dynamic lib. If I simply link these static libs to my app, no issues. However, there are a lot of multiple definition errors when the static libs are built to a dynamic lib with Qt Creator. Any solutions? Any build flag causes this problem?

Chris Kawa · 22 Dec 2023, 17:13

@JoeCFD The multiple definition error happens when at link stage you have the same symbols in multiple object files you try to link together. If the symbol in question is from one of your static libs it means you're linking that static lib multiple times, either directly or transitively.

A common case for this is when you have a structure like this:

(static lib X) -> (dynamic lib Y)
                       |
                       V
(static lib X) -> (executable)

but of course that's just one of many possible scenarios.

Start by drawing out a dependency graph of your entire solution and try to figure out where the doubled symbols occur. We can help with that, but you'll have to give more specifics about your app structure and how you link your objects.

JoeCFD · 22 Dec 2023, 17:32

@Chris-Kawa Thank for your reply, Chris. These static libs are third party ones and it is very messy to build them. Your structure is my case with a bunch of Xs.

This is something I got online:
When you link static libraries into an application, each library is typically independent, and the linker can resolve symbols without issues. However, when you combine these libraries into a single dynamic library, you may encounter naming conflicts.

A static lib is a collection of .o files. I may need to use ar to break it and remove all duplicate .o files. And then build rest of them into a dynamic one. Any better solution?

Chris Kawa · 22 Dec 2023, 17:53

@JoeCFD said:

I may need to use ar to break it and remove all duplicate .o files. And then build rest of them into a dynamic one. Any better solution?

Like I said - draw out the dependencies and reorganize your linking so that you link one library only once. If you're getting multiple definitions it means you're linking the same thing multiple times.

You really don't want to be fiddling with .o files. That's a maintenance nightmare.

Another common problem is when you have a structure like this:

(static lib X) -> (static lib A)  \
                                   (dynamic lib FOO)
(static lib X) -> (static lib B)  /

The multiple definitions problem happens because both static lib A and B contain symbols from X. To fix such issue don't link X into A or B, but directly to FOO, like this:

(static lib A)  \
(static lib X)  - (dynamic lib FOO)
(static lib B)  /

Static libs don't need to be self contained i.e. A doesn't need to contain symbols from X. It can and it makes it simpler if you only link A and FOO, but if multiple static libs contain X symbols you get your issue, and in that case it's better to break the dependencies apart.

JoeCFD · 22 Dec 2023, 18:06

@Chris-Kawa Thank you for your help, Chris. In my scenario, both static libs A and B contain symbols from X. Both A and B are static and linked to X already. How do I de-link X from A and B and add X to FOO directly?

'-Wl,--exclude-libs, X' is the right flag for removing X from A and B?

Chris Kawa · 22 Dec 2023, 18:27

@JoeCFD Do you have control over the compilation/linking of A and B or do you get them already linked? If it's the former just reconfigure your A and B projects not to link to X. If it's the latter then you might indeed need to temper with the resulting library.

The problem arises if X was configured differently for A and B. Libraries are often configurable with different defines, configs, switches and whatnot, so X.o linked to A might not be identical to X.o linked to B and it might not be as easy as just removing one of them. X.o from A lib might not work with B etc.
I don't know much about Linux, but on Windows it's a common issue when X is linked multiple times and brings in both static and dynamic versions of the CRT in the same binary. It leads to all kinds of nasty.

That's another argument for ensuring you link one library only once if you have that luxury, thus keeping everything compatible with each other and compiled with the same set of configurations.

JoeCFD · wrote on 22 Dec 2023, 18:27

@Chris-Kawa Unluckily, not, Chris. A, B and X are downloaded. If I can rebuild A and B, I will be able to build A, B and X to FOO easily. One way out may be to exclude X from the build with '-Wl,--exclude-libs, X', but break X and add its obj files to FOO while keeping A and B