Skip to content

[enhance] include what you use

Maxence Naud requested to merge upd_iwyu into dev

Context

Update core includes automaticaly using include-what-you-use to reduce compilation time following aidge#321 recommandations for using more forward declarations and cleaning includes.

I doubt it is possible to automate this process for now as the generated files do not compile. Some hand-made modifications remain for a couple of forward declarations that should be includes.

The mapping I used for pointing to Pybind instead of CPython is this one:

[
  { "include": ["<cpython/.*\\.h>",  "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<internal/.*\\.h>", "private", "<pybind11/pybind11.h>", "public"] },

  { "include": ["<object.h>",          "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<abstract.h>",        "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<bytesobject.h>",     "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<bytearrayobject.h>", "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<unicodeobject.h>",   "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<longobject.h>",      "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<floatobject.h>",     "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<complexobject.h>",   "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<dictobject.h>",      "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<listobject.h>",      "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<tupleobject.h>",     "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<setobject.h>",       "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<memoryobject.h>",    "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<weakrefobject.h>",   "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<frameobject.h>",     "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<genobject.h>",       "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<funcobject.h>",      "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<methodobject.h>",    "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<moduleobject.h>",    "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<import.h>",          "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<pydebug.h>",         "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<pylifecycle.h>",     "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<structmember.h>",    "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<traceback.h>",       "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<typeslots.h>",       "private", "<pybind11/pybind11.h>", "public"] },

  { "include": ["<pybind11/detail/.*\\.h>", "private", "<pybind11/pybind11.h>", "public"] },

  { "include": ["<pybind11/common.h>", "private", "<pybind11/pybind11.h>", "public"] },

  { "include": ["<pybind11/pytypes.h>",   "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<pybind11/cast.h>",      "private", "<pybind11/pybind11.h>", "public"] },
  { "include": ["<pybind11/attr.h>",      "private", "<pybind11/pybind11.h>", "public"] }
]

The compilation was performed with Clang++-21 and here is the iwyu line:

python <iwyu-directory>/iwyu_tool.py -p aidge/aidge_core/build_bundle_cpp/ -- -Xiwyu --mapping_file="$(pwd)/python-internals.imp" -Xiwyu --no_fwd_decls | python <iwyu-directory>/fix_includes.py --comments --update_comments --nosafe_headers --reorder

Tests

Computations benchmarked using hyperfine.

Command (iwyu) Mean [s] Min [s] Max [s] ratio
./setup.sh -m core --release --cxx clang++-21 16.470 ± 0.208 16.108 16.782 0.93
./setup.sh -m core -p --release --cxx clang++-21 78.966 ± 3.770 75.810 85.870 0.97
./setup.sh -m core --tests --release --cxx clang++-21 22.333 ± 0.859 21.654 24.458 0.96
Command (old) Mean [s] Min [s] Max [s]
./setup.sh -m core --release --cxx clang++-21 17.667 ± 0.206 17.282 18.004
./setup.sh -m core -p --release --cxx clang++-21 81.292 ± 3.539 76.917 86.072
./setup.sh -m core --tests --release --cxx clang++-21 23.340 ± 0.551 22.759 24.375
Edited by Maxence Naud

Merge request reports

Loading