Inline value for array access with variable key

[bmewburn]

I'm implementing an inline value provider in intelephense. When returning an InlineValueEvaluatableExpression for an array access expression with a variable for the key, the inline value is that of the array instead of the array element. Is it possible to resolve the element value here? If so, where in the code would I look if I was to attempt a PR?

<?php

$a = ['x', 'y', 'z'];
foreach ($a as $k => $v) {
    echo $a;
    echo $k;
    echo $v;
    echo $a[0];
    echo $a[$k];
    echo 1;
}

The problem seems similar to #1077 . I am running version 1.38.2.

[zobo]

Hi Ben.
Great to see you are implementing this in your LS. Indeed this is a known problem that I'm just revisiting due to the recent changes I did. I'm not 100% sure what evaluateRequest context is used by VSCode when it's resolving inline values. Would you be able to make a log file? (launch.json...log:true).
The reason is that I probably use Xdebug property_get instead of eval and Xdebug does strange things. As said this is a problem that recently came back on my table and seeing this input is very valuable. Thanks!

If you are playing with the source code of this extension, you want to look at this line. See what context is reported by vscode and switch from tryPropertyGet to tryEval. This does have other implications so it's not as easy as one would hope. I also don't want to stuff too much PHP language awareness in the extension...

On that note, if you are doing debugging related extensions to your LS, consider adding EvaluatableExpression support. I added this to vscode-php-instellisense and vscode-phpactor a while back. This isn't a standard LSP message, but rather something that you'd want to delegate to a LS. Look at the boilerplate code here and here. Didn't check if you already have this, so sorry if you do.

[bmewburn]

Thanks for the info @zobo .

I've added a log below. After testing more, I'm just going to return ranges for simple variables, at least for the first iteration. I think it can look too cluttered when properties and array access are shown inline too.

For the EvaluatableExpressionProvider what kind of expressions should be returned? I presume simple variables and property access expressions. What about call expressions? If the debugger is stopped at a breakpoint before a call that mutates some object and it is evaluated by hovering 3 times does this affect the state when continuing? Or does the debugger resume at the state when it first stopped at the breakpoint?

debugger log

<?php

$a = ['x', 'y', 'z'];
foreach ($a as $k => $v) {
    echo $a[$k];
    echo 1;
}

-> continueRequest
{ command: 'continue', arguments: { threadId: 1 }, type: 'request', seq: 14 }

<- continueResponse
Response {
  seq: 0,
  type: 'response',
  request_seq: 14,
  command: 'continue',
  success: true,
  body: { allThreadsContinued: false } }

xd(1) <- run -i 31
<- outputEvent
OutputEvent { seq: 0, type: 'event', event: 'output', body: { category: 'stdout', output: 'x' } }

x
xd(1) -> <?xml version="1.0" encoding="iso-8859-1"?><response xmlns="urn:debugger_protocol_v1" xmlns:xdebug="https://xdebug.org/dbgp/xdebug" command="run" transaction_id="31" status="break" reason="ok"><xdebug:message filename="file:///xxx/debugger.php" lineno="6"></xdebug:message></response>
<- stoppedEvent
StoppedEvent {
  seq: 0,
  type: 'event',
  event: 'stopped',
  body: { reason: 'breakpoint', threadId: 1, allThreadsStopped: false } }

-> threadsRequest
{ command: 'threads', type: 'request', seq: 15 }

<- threadsResponse
Response {
  seq: 0,
  type: 'response',
  request_seq: 15,
  command: 'threads',
  success: true,
  body: { threads: [ Thread { id: 1, name: 'Request 1 (10:36:04)' } ] } }

-> stackTraceRequest
{ command: 'stackTrace',
  arguments: { threadId: 1, startFrame: 0, levels: 20 },
  type: 'request',
  seq: 16 }

xd(1) <- stack_get -i 32
xd(1) -> <?xml version="1.0" encoding="iso-8859-1"?><response xmlns="urn:debugger_protocol_v1" xmlns:xdebug="https://xdebug.org/dbgp/xdebug" command="stack_get" transaction_id="32"><stack where="{main}" level="0" type="file" filename="file:///xxx/debugger.php" lineno="6"></stack></response>
<- stackTraceResponse
Response {
  seq: 0,
  type: 'response',
  request_seq: 16,
  command: 'stackTrace',
  success: true,
  body:
   { totalFrames: 1,
     stackFrames:
      [ { id: 3,
          name: '{main}',
          source: { name: 'debugger.php', path: '/xxx/debugger.php' },
          line: 6,
          column: 1 } ] } }

-> scopesRequest
{ command: 'scopes', arguments: { frameId: 3 }, type: 'request', seq: 17 }

xd(1) <- context_names -i 33 -d 0
xd(1) -> <?xml version="1.0" encoding="iso-8859-1"?><response xmlns="urn:debugger_protocol_v1" xmlns:xdebug="https://xdebug.org/dbgp/xdebug" command="context_names" transaction_id="33"><context name="Locals" id="0"></context><context name="Superglobals" id="1"></context><context name="User defined constants" id="2"></context></response>
<- scopesResponse
Response {
  seq: 0,
  type: 'response',
  request_seq: 17,
  command: 'scopes',
  success: true,
  body:
   { scopes:
      [ Scope { name: 'Locals', variablesReference: 4, expensive: false },
        Scope { name: 'Superglobals', variablesReference: 5, expensive: false },
        Scope { name: 'User defined constants', variablesReference: 6, expensive: false } ] } }

-> variablesRequest
{ command: 'variables',
  arguments: { variablesReference: 4 },
  type: 'request',
  seq: 18 }

xd(1) <- context_get -i 34 -d 0 -c 0
xd(1) -> <?xml version="1.0" encoding="iso-8859-1"?><response xmlns="urn:debugger_protocol_v1" xmlns:xdebug="https://xdebug.org/dbgp/xdebug" command="context_get" transaction_id="34" context="0"><property name="$a" fullname="$a" type="array" children="1" numchildren="3" page="0" pagesize="100"><property name="0" fullname="$a[0]" type="string" size="1" encoding="base64"><![CDATA[eA==]]></property><property name="1" fullname="$a[1]" type="string" size="1" encoding="base64"><![CDATA[eQ==]]></property><property name="2" fullname="$a[2]" type="string" size="1" encoding="base64"><![CDATA[eg==]]></property></property><property name="$k" fullname="$k" type="int"><![CDATA[0]]></property><property name="$v" fullname="$v" type="string" size="1" encoding="base64"><![CDATA[eA==]]></property></response>
<- variablesResponse
Response {
  seq: 0,
  type: 'response',
  request_seq: 18,
  command: 'variables',
  success: true,
  body:
   { variables:
      [ { name: '$a',
          value: 'array(3)',
          type: 'array',
          variablesReference: 7,
          presentationHint: {},
          evaluateName: '$a',
          indexedVariables: 3 },
        { name: '$k',
          value: '0',
          type: 'int',
          variablesReference: 0,
          presentationHint: {},
          evaluateName: '$k',
          indexedVariables: undefined },
        { name: '$v',
          value: '"x"',
          type: 'string',
          variablesReference: 0,
          presentationHint: {},
          evaluateName: '$v',
          indexedVariables: undefined } ] } }

-> evaluateRequest
{ command: 'evaluate',
  arguments: { expression: '$a[$k]', frameId: 3, context: 'watch' },
  type: 'request',
  seq: 19 }

xd(1) <- context_names -i 35 -d 0
xd(1) -> <?xml version="1.0" encoding="iso-8859-1"?><response xmlns="urn:debugger_protocol_v1" xmlns:xdebug="https://xdebug.org/dbgp/xdebug" command="context_names" transaction_id="35"><context name="Locals" id="0"></context><context name="Superglobals" id="1"></context><context name="User defined constants" id="2"></context></response>
xd(1) <- property_get -i 36 -d 0 -c 0 -n "$a[$k]"
xd(1) -> <?xml version="1.0" encoding="iso-8859-1"?><response xmlns="urn:debugger_protocol_v1" xmlns:xdebug="https://xdebug.org/dbgp/xdebug" command="property_get" transaction_id="36"><property name="$a[$k]" fullname="$a[$k]" type="array" children="1" numchildren="3" page="0" pagesize="100"><property name="0" fullname="$a[$k][0]" type="string" size="1" encoding="base64"><![CDATA[eA==]]></property><property name="1" fullname="$a[$k][1]" type="string" size="1" encoding="base64"><![CDATA[eQ==]]></property><property name="2" fullname="$a[$k][2]" type="string" size="1" encoding="base64"><![CDATA[eg==]]></property></property></response>
<- evaluateResponse
Response {
  seq: 0,
  type: 'response',
  request_seq: 19,
  command: 'evaluate',
  success: true,
  body: { result: 'array(3)', variablesReference: 8 } }

[zobo]

Interesting, it uses the watch context. I guess I'll have to build in at least some basic expression parsing to catch things like $array[$key] and in that case not use property_get but rather directly fall back to eval.

The other topic, EvaluatableExpressionProvider is used to help VSCode understand the language structure when it tries to figure out what to ask the debugger to evaluate in a hover scenario. To put it simpler, VSCode gives you a document position where the user is hovering, and is asking you "is there a variable under this position that I can send to debugger for evaluation". Here are some simple test cases I implemented in phpactor.

[bmewburn]

Sorry to pester with more questions, but the evaluatable expression provider parameters don't provide any information on where the debugger might be stopped. So hovering over a variable/parameter that is not in scope but has the same name as an in scope variable shows that value even if it is irrelevant. For example hovering over the $a parameter in function testScope while stopped in the foreach loop below. Can this be avoided somehow? I wonder if it would be better to send another inlineValues request here, instead of evaluatableExpression, with a 0 length range (start and end are the hover position), and then in combination with the stoppedLocation the LS can determine if it is currently in scope.

<?php

$a = ['x', 'y', 'z', 'nested' => [1, 2, 3]];
foreach ($a as $k => $v) {
    print_r($a[$k]);
    echo 1; // <- stopped here
}

function testScope($a) { // <- hover here
    var_dump($a);
}

[zobo]

That is a very interesting observation. I have, until now, brushed this of as - well there is no context to figure out what variables are in scope and what not, so this means that EvaluatableExpression just answers the question of hover from a static analysis point of view: "Is the verb at this position a variable or not? Ex: I'm hovering over abc, but look, this is actually part of a variable, here eval this please: $this->data[2]->abc".

However, thinking about it, if we could know where the current execution is stopped, and using the AST we could determine what hover positions are in scope or not...

The execution position is par of the DAP stack trace and one can get the activeStackItem but I do not see a way for an extension to ask the IDE "where is the current code execution stopped?". Short of agreeing upon a customRequest the LS could send to the DA.

Maybe this is why EvaluatableExpression isn't part of LSP, as it requires so much interaction between a debugger and a language server...

I think I'll open an issue with in the vscode repo.

Do I understand your question correctly?

[bmewburn]

Yes that's correct. It doesn't seem like it should be the role of the LS to decide if the hover is relevant or not, maybe it should just answer the question about whether it's a variable. Though, if xdebug can not provide info about the range of the stopped scope so that the DAP extension itself can determine if the hover position is relevant then the LS probably needs to do it.

Another factor is what kind of expression it is. Does the DAP extension or the LS decide if the expression should be evaluated or not? For example should $obj->mutatesObj()->prop be evaluated if it changes the state of $obj? And then evaluating even simple expressions like $obj['prop'] or $obj->prop could cause side effects when considering ArrayAccess and __get.

[bmewburn]

So if I understand correctly, to do this using the existing APIs available I would register an evaluatable expression provider in the intelephense vscode client. When a request comes through to the provider:

1. query the vscode api for the active stack item.
2. query vscode-debug for the stack trace (is that possible via vscode.commands.executecommand ? Is debug.stackTrace the cmd?)
3. Search the result for the stack frame by id and get the position.
4. Send to intelephense a custom request or reuse inlineValues and send the context (stoppedLocation) of the request.

Would that work?

[zobo]

Yes, roughly. But not sure what communication would be required between the two extensions. As the stack trace information is only preset inside the debug adapter, and the extension itself does not have this information. But VSCode does. That's why I opened the question with Microsoft. Let's give them a few days and we'll see where we are.