DAOS-18304 ddb: Add unit test to ddb code#17967
Draft
knard38 wants to merge 1 commit intockochhof/fix/master/daos-18291from
Draft
DAOS-18304 ddb: Add unit test to ddb code#17967knard38 wants to merge 1 commit intockochhof/fix/master/daos-18291from
knard38 wants to merge 1 commit intockochhof/fix/master/daos-18291from
Conversation
|
Ticket title is 'Add unit test to ddb go code' |
Collaborator
|
Test stage Unit Test completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17967/1/testReport/ |
aadfb16 to
92d3255
Compare
Collaborator
|
Test stage Unit Test completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17967/3/testReport/ |
92d3255 to
b2a2966
Compare
Collaborator
|
Test stage Functional on EL 9 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17967/4/execution/node/1032/log |
Refactoring DDB code to allow unit testing of the go part code: - The GO interface DdbApi is introduced to allow the use of GO test stub. - Refactor initialisation of the C structure from the Go Code to allow the use of GO test stub. - The useless C attributes dc_pool_path and dc_db_path are removed. Following issues are also fixed: - Remove invalid restrictions on pool file path size. - Improper cleanup of the function ddb_run_feature(). - Fix invalid return code with non-interactive mode - Fix rm_pool command: - Add db_path parameter to pool destroy command. - Makes vos_path parameter mandatory to pool destroy command. Features: recovery Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>
b2a2966 to
2ff60fd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR introduces a
DdbApiGo interface to decouple the DDB commandlayer from the C VOS implementation, making the Go code unit-testable
with stubs that do not require a live VOS environment. Several bugs
discovered during this refactoring are also fixed.
Refactoring
Go interface and method conversion
DdbApiinterface so a Go test stub can be injected inplace of the real CGo implementation.
ddbXxx(ctx *DdbContext, ...)free functions to(ctx *DdbContext) Xxx(...)methods implementingDdbApi.DdbApithroughaddAppCommands(),createGrumbleApp()and
parseOpts()instead of passing*DdbContextdirectly.DdbContext lifecycle
Init()to a method on a caller-allocatedDdbContextsothe object lifecycle is fully controlled by the caller (or a test
stub), instead of being allocated inside
InitDdb().Removal of dc_pool_path and dc_db_path
dc_pool_pathanddc_db_pathfields fromstruct ddb_ctx. These were workarounds used to pass path stringsfrom Go to C by storing them directly in the context struct. They are
now passed as explicit function arguments at every call site, which
is cleaner and removes the need for the
SetCString()helper.parseOpts improvements
prefixesToSkip+HasPrefixloop with anoAutoOpenmap for exact command-name matching, avoiding accidental prefix
collisions (e.g. a future command whose name starts with an existing
command name).
os.Exit()call insideparseOpts()when printing help:printHelp()now returns normally andparseOpts()returnsnil,keeping the function testable.
Bug Fixes
Invalid command error messages
unknownCmdErrorstruct to replace raw stringcomparison against
grumbleUnknownCmdErr. This enables type-safedetection in
exitWithError().exitWithError()now prints the available command list when theerror is an
unknownCmdError, making it immediately clear to theuser what commands are available.
exitWithError(): fix duplicatedERROR:prefix line that waspreviously printed when a fault resolution was available.
printCmdHelp(): when--helpis requested for an unknown command,the error message and the command list are now printed together via
exitWithError(&unknownCmdError{}), consistent with all otherinvalid-command paths.
ddb_run_feature() state corruption (C)
dc_pohanddc_write_modewere reset unconditionally on exit,corrupting pool state when the pool was already open before the
command ran. The reset is now guarded by the
closeflag.dv_pool_open()fails.Non-interactive mode error propagation
runCmdStr()andrunFileCmds()were previously loggedand discarded. They are now properly returned to the caller and
reported via
exitWithError().Startup ordering
debug.SetTraceback("crash")fromparseOpts()tomain()soit runs unconditionally before any other code, including
DisableCStdoutBuffering().--version flag
--versionnow prints directly and returns instead of synthesisinga spurious
versioncommand, which previously bypassed commandvalidation.
--db_path without --vos_path
--db_pathwithout--vos_pathwas silently ignored. Itnow returns an explicit error.
Pool close cleanup
closePoolIfOpen()helper.defer closePoolIfOpen()when auto-opening via--vos_pathtoguarantee cleanup on all exit paths.
Go wrapper memory leaks and ordering
Feature(): C strings for theenableanddisableflagarguments were passed anonymously without being freed. They are now
assigned to named variables and freed via
defer.DtxStat():defer freeString(options.path)was placed afteroptions.detailsassignment. It is now placed immediately after theCStringallocation, consistent with all other wrappers.rm_pool Command
--vos_pathmandatory: it was previously optional, causing acrash when omitted.
db_pathexplicitly throughout the call chain:RmPool()→ddb_run_rm_pool()→dv_pool_destroy().feature Command
db_pathexplicitly throughout the call chain:Feature()→ddb_run_feature()→dv_pool_open().opencommand also now passesdb_pathdirectly as a function argument instead of storing it in the context.
VOS Path Length Restriction Removed
DB_PATH_SIZEandVOS_PATH_SIZEfrom 256 toPATH_MAX,removing the artificial restriction on deep filesystem hierarchies.
vf_vos_file_path[VOS_PATH_SIZE]tostruct vos_file_partsandpopulate it in
parse_vos_file_parts()so the full original path ispreserved for
vos_pool_open()/vos_pool_destroy_ex().vos_pathanddb_pathinddb_parse_tests.c.dv_pool_open / dv_pool_destroy API
(path, path_parts, poh, flags, write_mode)to
(path, db_path, ctx, flags), movingvos_file_partsallocationinto a new internal
create_vos_file_parts()helper inddb_vos.c.ddb_commands.cand the test suite are updatedaccordingly.
Test Fixes (C)
dump_dtx_cmd_tests()intodcv_suit_setup()so all DTX tests share a consistent fixture.ls,value_dumpandilog_dumptests to match the actual test data layout after DTX record insertion
was moved to suite setup.
dtx_statregex to match the new"DTX entries statistics of the pool:"message (removes the staledc_pool_pathnull-pointer reference in the format string).Steps for the author:
After all prior steps are complete: