Skip to content

Adjust the standards for proto target to align with those of Protocol Buffers.#3800

Open
owent wants to merge 5 commits intoopen-telemetry:mainfrom
owent:proto_std_should_follow_pb
Open

Adjust the standards for proto target to align with those of Protocol Buffers.#3800
owent wants to merge 5 commits intoopen-telemetry:mainfrom
owent:proto_std_should_follow_pb

Conversation

@owent
Copy link
Copy Markdown
Member

@owent owent commented Jan 6, 2026

Fixes #3799

Changes

Please provide a brief description of the changes here.

For significant contributions please make sure you have completed the following items:

  • CHANGELOG.md updated for non-trivial changes
  • Unit tests have been added
  • Changes in public API reviewed

Analysis

The generated .pb.cc codes

inline constexpr EntityRef::Impl_::Impl_(
    ::_pbi::ConstantInitialized) noexcept
      : id_keys_{},
        description_keys_{},
        schema_url_(
            &::google::protobuf::internal::fixed_address_empty_string,
            ::_pbi::ConstantInitialized()),
        type_(
            &::google::protobuf::internal::fixed_address_empty_string,
            ::_pbi::ConstantInitialized()),
        _cached_size_{0} {}

port.h in protobuf:

// Take advantage of C++20 constexpr support in std::string.
class alignas(8) GlobalEmptyStringConstexpr {
 public:
  const std::string& get() const { return value_; }
  // Nothing to init, or destroy.
  std::string* Init() const { return nullptr; }

  template <typename T = std::string, bool = (T(), true)>
  static constexpr std::true_type HasConstexprDefaultConstructor(int) {
    return {};
  }
  static constexpr std::false_type HasConstexprDefaultConstructor(char) {
    return {};
  }

 private:
  std::string value_;
};

using GlobalEmptyString = std::conditional_t<
    GlobalEmptyStringConstexpr::HasConstexprDefaultConstructor(0),
    const GlobalEmptyStringConstexpr, GlobalEmptyStringDynamicInit>;

PROTOBUF_EXPORT extern GlobalEmptyString fixed_address_empty_string;

port.cc in protobuf

PROTOBUF_ATTRIBUTE_NO_DESTROY PROTOBUF_CONSTINIT
    PROTOBUF_ATTRIBUTE_INIT_PRIORITY1 GlobalEmptyString
        fixed_address_empty_string{};

If protobuf is built with C++17 but the generated .pb.cc files are built with C++20, protobuf exports GlobalEmptyStringDynamicInit fixed_address_empty_string; while .pb.cc expects GlobalEmptyStringConstexpr fixed_address_empty_string;, causing an unresolved external symbol.

On GCC/Clang, constexpr typically doesn’t require an extra symbol, so this mismatch doesn’t surface; MSVC’s behavior is right because constexpr is not a force rule, but it exposes the issue.

This didn’t happen before v30 because only the dynamic GlobalEmptyString existed. It was fixed in protobuf v32 by always using GlobalEmptyStringDynamicInit on MSVC, so the only affected case is MSVC with protobuf v31.

Copilot AI review requested due to automatic review settings January 6, 2026 12:35
@owent owent requested a review from a team as a code owner January 6, 2026 12:35
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds CI validation to ensure OpenTelemetry C++ can be built with a different C++ standard than the protobuf dependency it uses, addressing issue #3799 where MSVC linking errors occurred when protobuf was compiled with C++17 but OpenTelemetry was built with C++20.

Key Changes:

  • Adds new cmake.different_std.test CI target for both Linux (GCC) and Windows (MSVC) platforms
  • Linux GCC test installs dependencies with C++17, then builds OpenTelemetry with C++23
  • Windows MSVC test validates similar scenario with C++20 for OpenTelemetry

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File Description
ci/do_ci.sh Adds new cmake.different_std.test target for Linux/GCC that builds with C++23 STL and enables OTLP file support
ci/do_ci.ps1 Adds new cmake.different_std.test target for Windows/MSVC that builds with C++20 STL, maintainer mode, and OTLP file support
.github/workflows/ci.yml Adds two new CI jobs: one for GCC (installs protobuf with C++17, builds with C++23) and one for MSVC (tests different C++ standards)

@codecov
Copy link
Copy Markdown

codecov bot commented Jan 6, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.12%. Comparing base (508ec67) to head (d470150).

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #3800   +/-   ##
=======================================
  Coverage   90.12%   90.12%           
=======================================
  Files         227      227           
  Lines        7261     7261           
=======================================
  Hits         6543     6543           
  Misses        718      718           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@owent owent force-pushed the proto_std_should_follow_pb branch 4 times, most recently from 3f725c7 to 7cab857 Compare January 10, 2026 17:00
@owent owent changed the title [WIP] Adjust the standards for proto target to align with those of Protocol Buffers. Adjust the standards for proto target to align with those of Protocol Buffers. Jan 13, 2026
- name: run otprotocol test
run: ./ci/do_ci.ps1 cmake.exporter.otprotocol.with_async_export.test

# Test case where protobuf is built with C++17 and OpenTelemetry with C++20 on MSVC will trigger a Internal compiler error
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove this commented out test?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test currently triggers an MSVC bug and causes an internal compiler error. Should we enable it once the MSVC compiler on GitHub runners is updated? I’m leaving it here so we can re-enable it quickly. It’s the only test that reproduces #3799; I can reproduce it locally with VS 2026, but that isn’t usable on gihutb right now. Alternatively, should we remove it and add it back later?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the ICE? Probably add TODO to track it?

Copy link
Copy Markdown
Member Author

@owent owent Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logs can be found at https://github.com/open-telemetry/opentelemetry-cpp/actions/runs/20818142168/job/59799024061

When building protobuf v31.1

libupb.vcxproj -> C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf-build\bin\Debug\libupbd.lib
    Building Custom Rule C:/Users/runneradmin/AppData/Local/Temp/otel-cpp-third-party-build/external/protobuf/src/protobuf/CMakeLists.txt
    any.cc
    any_lite.cc
    arena.cc
    arena_align.cc
    arenastring.cc
    arenaz_sampler.cc
    code_generator.cc
    code_generator_lite.cc
    plugin.pb.cc
    retention.cc
    cpp_features.pb.cc
    descriptor.cc
C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf\src\google/protobuf/descriptor_visitor.h(108,1): fatal error C1001: Internal compiler error. [C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf-build\protoc-gen-upb.vcxproj] [C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\protobuf-build.vcxproj]
    (compiler file 'msc1.cpp', line 1589)
     To work around this problem, try simplifying or changing the program near the locations listed above.
    If possible please provide a repro here: https://developercommunity.visualstudio.com/
    Please choose the Technical Support command on the Visual C++
     Help menu, or open the Technical Support help file for more information
  C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf\src\google/protobuf/descriptor_visitor.h(108,1): message : the template instantiation context (the oldest one first) is [C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf-build\protoc-gen-upb.vcxproj]
  C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf\src\google\protobuf\descriptor.cc(8380,13): message : see reference to function template instantiation 'void google::protobuf::internal::VisitDescriptors<google::protobuf::DescriptorBuilder::CheckVisibilityRules::<lambda_eb886289529f14dfd0500371ca9c46d6>>(const google::protobuf::FileDescriptor &,const google::protobuf::FileDescriptorProto &,Visitor)' being compiled [C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf-build\protoc-gen-upb.vcxproj]
            with
            [
                Visitor=google::protobuf::DescriptorBuilder::CheckVisibilityRules::<lambda_eb886289529f14dfd0500371ca9c46d6>
            ]
C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf\src\google/protobuf/descriptor_visitor.h(167,58): message : see reference to function template instantiation 'void google::protobuf::internal::VisitImpl<google::protobuf::internal::VisitDescriptors::VisitorImpl>::Visit<const google::protobuf::FileDescriptorProto>(const google::protobuf::FileDescriptor &,const google::protobuf::FileDescriptorProto &)' being compiled [C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf-build\protoc-gen-upb.vcxproj]
  C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf\src\google/protobuf/descriptor_visitor.h(109,5): message : while compiling class template member function 'unknown-type google::protobuf::DescriptorBuilder::CheckVisibilityRules::<lambda_eb886289529f14dfd0500371ca9c46d6>::operator ()(const _T1 &,const _T2 &) const' [C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf-build\protoc-gen-upb.vcxproj]
    INTERNAL COMPILER ERROR in 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\bin\HostX64\x64\CL.exe'
        Please choose the Technical Support command on the Visual C++
        Help menu, or open the Technical Support help file for more information
    Building Custom Rule C:/Users/runneradmin/AppData/Local/Temp/otel-cpp-third-party-build/external/protobuf/src/protobuf/CMakeLists.txt
    any.cc
    any_lite.cc
    arena.cc
    arena_align.cc
    arenastring.cc
    arenaz_sampler.cc
    code_generator.cc
    code_generator_lite.cc
    plugin.pb.cc
    retention.cc
    cpp_features.pb.cc
    descriptor.cc
C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf\src\google/protobuf/descriptor_visitor.h(108,1): fatal error C1001: Internal compiler error. [C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf-build\protoc-gen-upbdefs.vcxproj] [C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\protobuf-build.vcxproj]
    (compiler file 'msc1.cpp', line 1589)
     To work around this problem, try simplifying or changing the program near the locations listed above.
    If possible please provide a repro here: https://developercommunity.visualstudio.com/
    Please choose the Technical Support command on the Visual C++
     Help menu, or open the Technical Support help file for more information
  C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf\src\google/protobuf/descriptor_visitor.h(108,1): message : the template instantiation context (the oldest one first) is [C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf-build\protoc-gen-upbdefs.vcxproj]
  C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf\src\google\protobuf\descriptor.cc(8380,13): message : see reference to function template instantiation 'void google::protobuf::internal::VisitDescriptors<google::protobuf::DescriptorBuilder::CheckVisibilityRules::<lambda_eb886289529f14dfd0500371ca9c46d6>>(const google::protobuf::FileDescriptor &,const google::protobuf::FileDescriptorProto &,Visitor)' being compiled [C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf-build\protoc-gen-upbdefs.vcxproj]
            with
            [
                Visitor=google::protobuf::DescriptorBuilder::CheckVisibilityRules::<lambda_eb886289529f14dfd0500371ca9c46d6>
            ]
  C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf\src\google/protobuf/descriptor_visitor.h(167,58): message : see reference to function template instantiation 'void google::protobuf::internal::VisitImpl<google::protobuf::internal::VisitDescriptors::VisitorImpl>::Visit<const google::protobuf::FileDescriptorProto>(const google::protobuf::FileDescriptor &,const google::protobuf::FileDescriptorProto &)' being compiled [C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf-build\protoc-gen-upbdefs.vcxproj]
  C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf\src\google/protobuf/descriptor_visitor.h(109,5): message : while compiling class template member function 'unknown-type google::protobuf::DescriptorBuilder::CheckVisibilityRules::<lambda_eb886289529f14dfd0500371ca9c46d6>::operator ()(const _T1 &,const _T2 &) const' [C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\protobuf\src\protobuf-build\protoc-gen-upbdefs.vcxproj]
    INTERNAL COMPILER ERROR in 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\bin\HostX64\x64\CL.exe'
        Please choose the Technical Support command on the Visual C++
        Help menu, or open the Technical Support help file for more information
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(254,5): error MSB8066: Custom build for 'C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\CMakeFiles\b85673c8d2c887062bdd371d2afff3e1\protobuf-mkdir.rule;C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\CMakeFiles\b85673c8d2c887062bdd371d2afff3e1\protobuf-download.rule;C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\CMakeFiles\b85673c8d2c887062bdd371d2afff3e1\protobuf-update.rule;C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\CMakeFiles\b85673c8d2c887062bdd371d2afff3e1\protobuf-patch.rule;C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\CMakeFiles\b85673c8d2c887062bdd371d2afff3e1\protobuf-configure.rule;C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\CMakeFiles\b85673c8d2c887062bdd371d2afff3e1\protobuf-build.rule;C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\CMakeFiles\78a33ba8e4d1644ab3e8d5cd79f88cfd\protobuf-build.rule;D:\a\opentelemetry-cpp\opentelemetry-cpp\install\cmake\CMakeLists.txt' exited with code 1. [C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\protobuf-build.vcxproj]
  Performing install step for 'curl'
  MSBuild version 17.14.23+b0019275e for .NET Framework
  
    1>Checking Build System
    Building Custom Rule C:/Users/runneradmin/AppData/Local/Temp/otel-cpp-third-party-build/external/curl/src/curl/lib/CMakeLists.txt
    libcurl_object.vcxproj -> C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\curl\src\curl-build\lib\libcurl_object.dir\Debug\libcurl_object.lib
    Building Custom Rule C:/Users/runneradmin/AppData/Local/Temp/otel-cpp-third-party-build/external/curl/src/curl/lib/CMakeLists.txt
    libcurl_shared.vcxproj -> C:\Users\runneradmin\AppData\Local\Temp\otel-cpp-third-party-build\external\curl\src\curl-build\lib\Debug\libcurl-d.dll
    Building Custom Rule C:/Users/runneradmin/AppData/Local/Temp/otel-cpp-third-party-build/external/curl/src/curl/CMakeLists.txt
    1>
    -- Install configuration: "Debug"

I successfully built and tested Protobuf v31.1 on my system. It works well with the latest versions of both VS 2022 and VS 2026.
TODO comment is added and I squashed commits to clean up the history.

@owent owent force-pushed the proto_std_should_follow_pb branch from 5426ace to 4e79eb3 Compare January 29, 2026 02:41
Fixes third_party_latest PATH

install latest third party libs for windows

Fixes copyright and parallel

Fixes cmake args

Restore windows version

try to reproduce link error

add --build-shared-libs "ON"

Allow install_thirdparty to set BUILD_SHARED_LIBS

Make the proto  targets follow the CXX_STANDARD using by protobuf

Fixes the compatibility for old version of protobuf with C++14 used

Fixes DEFINED CMAKE_CXX_STANDARD checking

Fixes compiling

Fixes spelling

Patch protobuf v31 only is enough

Fixes cmake var

Add TODO
@owent owent force-pushed the proto_std_should_follow_pb branch from ed30cbb to 6929949 Compare February 5, 2026 02:32
@owent owent requested a review from ThomsonTan February 6, 2026 10:58
AND CMAKE_CXX_STANDARD GREATER_EQUAL 20
AND (Protobuf_VERSION_MAJOR EQUAL 31 OR Protobuf_VERSION MATCHES
"31\\..*"))
set(protobuf_lib_compile_features_cxx_std 17)
Copy link
Copy Markdown
Member

@dbarker dbarker Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to consider users building protobuf and abseil-cpp with other c++ standards (c++14, 20, 23) and would they also experience similar ABI issues on other OSes?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This issue was introduced in protobuf v31 and has already been fixed in protobuf v32.
Protobuf v31 requires C++17, and the ABI remains compatible when it is built with C++20 or later.
GCC and Clang may inline the relevant code, which can hide the issue. While this is more of a workaround than a proper fix, it should not introduce additional problems.

set(protobuf_lib_compile_features_cxx_std 17)
message(
STATUS
"protobuf::libprotobuf ${Protobuf_VERSION} detected and we force set CXX_STANDARD to 17 for .pb.cc files."
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would a simple option for the OTELCPP_PROTO_CXX_STANDARD address the original issue more generally?

# Default to what the rest of the project is using
set(OTELCPP_PROTO_CXX_STANDARD "${CMAKE_CXX_STANDARD}" CACHE STRING "The C++ standard used to compile the imported Protobuf libraries")

# Apply the user defined cxx standard to the proto target
set_target_properties(opentelemetry_proto PROPERTIES CXX_STANDARD ${OTELCPP_PROTO_CXX_STANDARD})

Copy link
Copy Markdown
Member Author

@owent owent Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the CMake documentation for LANG_STANDARD (https://cmake.org/cmake/help/latest/prop_tgt/LANG_STANDARD.html), the actual C++ standard used for compilation may be higher than the value specified by the CXX_STANDARD property.

Simply setting CXX_STANDARD does not work for our game server projects, so I changed implementation to patch the final compiling options.

@lalitb
Copy link
Copy Markdown
Member

lalitb commented Mar 25, 2026

@owent Thanks for the PR. The PR look much broader than the bug it is trying to fix. The actual issue appears to be narrow: MSVC + protobuf v31 + building the generated .pb.cc files with a higher C++ standard than protobuf itself. That should be fixable with a small change in cmake/opentelemetry-proto.cmake plus a simple CXX_STANDARD override on the generated proto targets in cmake/tools.cmake. I would suggest to narrow down this PR to minimal fix with

  • narrow detection in cmake/opentelemetry-proto.cmake
  • set CXX_STANDARD directly on opentelemetry_proto / opentelemetry_proto_grpc
  • avoid the regex-based compile flag rewriting and unrelated warning flag changes

And then we can have a separate infra PR with install_thirdparty.ps1/sh related changes if required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MSVC may link error when using protobuf built with C++17 but build otel-cpp with C++20

5 participants