cf-agent -Kf ./example.cf error: readjson: data error parsing JSON file '/tmp/mydata.json': No data error: readjson: data error parsing JSON file '/tmp/mydata.json': No data error: readjson: data error parsing JSON file '/tmp/mydata.json': No data error: readjson: data error parsing JSON file '/tmp/mydata.json': No data error: readjson: data error parsing JSON file '/tmp/mydata.json': No data error: readjson: data error parsing JSON file '/tmp/mydata.json': No data error: readjson: data error parsing JSON file '/tmp/mydata.json': No data error: readjson: data error parsing JSON file '/tmp/mydata.json': No data error: readjson: data error parsing JSON file '/tmp/mydata.json': No data error: readjson: data error parsing JSON file '/tmp/mydata.json': No data error: readjson: data error parsing JSON file '/tmp/mydata.json': No data error: readjson: data error parsing JSON file '/tmp/mydata.json': No data error: readjson: data error parsing JSON file '/tmp/mydata.json': No data

I have policy that puts a data file into place and reads data from it. Why do I get so many readjson() errors and how can I suppress them?

The minimum required to reproduce the error from readjson is:

12345
  bundle agent main
  {
      vars:
        "myvar" data => readjson( "/tmp/mydata.json", 1M);
  }

Now, what happens if we add a promise that manages the file?

 1 2 3 4 5 6 7 8 9101112
  body file control { inputs => {"$(sys.libdir)/files.cf"}; }
  bundle agent main
  {
      vars:
        "myvar" data => readjson( "/tmp/mydata.json", 1M);

      files:
        "/tmp/mydata.json"
          create => "true",
          edit_line => insert_lines( '{ "Hello": "world" }' ),
          edit_defaults => empty;
  }

We can run with verbose logging to see more details.

Snips for brevity ...


  cf-agent -Kvf /tmp/example.cf
   verbose:  CFEngine Core 3.11.0
   ...
   verbose: ----------------------------------------------------------------
   verbose:  Initialization preamble 
   verbose: ----------------------------------------------------------------
   ... 
   verbose: ----------------------------------------------------------------
   verbose:  Environment discovery 
   verbose: ----------------------------------------------------------------
   ...
   verbose: Verifying the syntax of the inputs...
   verbose: Checking policy with command '"/home/nickanderson/.cfagent/bin/cf-promises" -c "./example.cf"'
     error: readjson: data error parsing JSON file '/tmp/mydata.json': No data
     error: readjson: data error parsing JSON file '/tmp/mydata.json': No data
     error: readjson: data error parsing JSON file '/tmp/mydata.json': No data
     error: readjson: data error parsing JSON file '/tmp/mydata.json': No data
   verbose: Saved policy validated marker file '/home/nickanderson/.cfagent/state/cf_promises_validated'
 #+END_EXAMPLE

 The four above errors are emitted during syntax validation.

#+BEGIN_EXAMPLE
  verbose: ----------------------------------------------------------------
  verbose:  Loading policy 
  verbose: ----------------------------------------------------------------
  verbose: BEGIN parsing file: ./example.cf
  verbose: END   parsing file: ./example.cf
    error: readjson: data error parsing JSON file '/tmp/mydata.json': No data
    error: readjson: data error parsing JSON file '/tmp/mydata.json': No data
  verbose: BEGIN parsing file: /home/nickanderson/.cfagent/inputs/lib/files.cf
  verbose: END   parsing file: /home/nickanderson/.cfagent/inputs/lib/files.cf
  verbose: BEGIN parsing file: /home/nickanderson/.cfagent/inputs/lib/common.cf
  verbose: END   parsing file: /home/nickanderson/.cfagent/inputs/lib/common.cf
  verbose: Running full policy integrity checks

The above errors are emitted during the resolution that occurs during parsing inputs.


  verbose: ----------------------------------------------------------------
  verbose:  Preliminary variable/class-context convergence 
  verbose: ----------------------------------------------------------------
    error: readjson: data error parsing JSON file '/tmp/mydata.json': No data
    error: readjson: data error parsing JSON file '/tmp/mydata.json': No data
  verbose: string_mustache: argument 'default:main.myvar' does not resolve to a container or a list or a CFEngine array
  ...
  verbose: Skipping promise 'DEBUG $(this.bundle): $(link) will be a symlink to $(target)' because 'if'/'ifvarclass' is not defined
    error: readjson: data error parsing JSON file '/tmp/mydata.json': No data
    error: readjson: data error parsing JSON file '/tmp/mydata.json': No data

The above errors are emitted during pre-evaluation.


 verbose: Setting minimum acceptable TLS version: 1.0
 verbose: ----------------------------------------------------------------
 verbose:  Begin policy/promise evaluation 
 verbose: ----------------------------------------------------------------
 verbose: Using bundlesequence =>  {"main"}
 verbose: B: *****************************************************************
 verbose: B: BEGIN bundle main
 verbose: B: *****************************************************************
 verbose: V: .........................................................
 verbose: V: BEGIN variables (pass 1)
   error: readjson: data error parsing JSON file '/tmp/mydata.json': No data
 verbose: V:     Computing value of 'myvar'
   error: readjson: data error parsing JSON file '/tmp/mydata.json': No data
   error: readjson: data error parsing JSON file '/tmp/mydata.json': No data

The above errors are emitted during the first pass of variables during normal order (main evaluation).

After that the json data file is created.


 verbose: P: .........................................................
 verbose: P: BEGIN promise 'promise_example_cf_8' of type "files" (pass 1)
 verbose: P:    Promiser/affected object: '/tmp/mydata.json'
 verbose: P:    Part of bundle: main
 verbose: P:    Base context class: any
 verbose: P:    Stack path: /default/main/files/'/tmp/mydata.json'[1]
 verbose: Using literal pathtype for '/tmp/mydata.json'
 verbose: No mode was set, choose plain file default 0600
    info: Created file '/tmp/mydata.json', mode 0600

No further errors happen because now that the file exists it can be successfully parsed.

How can we suppress the errors?

You can guard the vars promise based on when there is a json file present, or based on the json_copy promise itself. But there are several things to consider. What is right depends on the specifics of the behavior you are looking for.

Considerations:

I tend to just base on the file presence, its the minimum necessary to suppress the errors:

123456
  bundle agent main
  {
      vars:
        "myvar" data => readjson( "/tmp/mydata.json", 1M)
          if => fileexists( "/tmp/mydata.json" );
  }

If you guard based on the copy_from promise being kept or repaired the variable will only populate if the agent can successfully verify that the file looks the same locally and remotely. Do you want to use stale data if you cant reach the server?

 1 2 3 4 5 6 7 8 91011121314151617
  bundle agent main
  {
    vars:

        # This will only trigger if the copy_from promise is KEPT or REPAIRED
        # If the server is unavailable for whatever reason and the remote client
        # is unable to verify then the variable will not be populated.

        "myvar"
          data => readjson( "/tmp/mydata.json", 1M),
          depends_on => { "ensure_data_up_to_date" };

    files:
        "/tmp/mydata.json"
          copy_from => remote_dcp( "/srv/mydata.json", $(sys.policy_hub) ),
          handle => "ensure_data_up_to_date";
  }

Perhaps you only want the variable populated if the copy_from promise has been attempted (regardless of success or failure), and that there is data on the disk, and that data is valid.

 1 2 3 4 5 6 7 8 910111213141516171819202122232425262728293031
  bundle agent main
  {
    vars:

      # Only load the json data if the json is valid

      valid_json::

        "myvar"
          data => readjson( "/tmp/mydata.json", 1M);

    classes:

      # Only validate the data if we have TRIED to update the data. We don't care
      # if it was successful or not, only that we tried.

      ensure_data_up_to_date_reached::

      # Validate with some external tool (no native function avaialable to simply test json).
      # python -m json.tool 
      # jq .

        "valid_json"  expression => returnszero( "/usr/bin/python -m json.tool /tmp/mydata.json", noshell);

    files:

        "/tmp/mydata.json"
          copy_from => remote_dcp( "/srv/mydata.json", $(sys.policy_hub) ),
          handle => "ensure_data_up_to_date",
          classes => results( "bundle", "ensure_data_up_to_date" );
  }