How does Intellisense work?
What is Intellisense?
Many modern editors have features to make life easier when working code, markup, and other types of structured text. Collectively, Microsoft calls these features “Intellisense” and they include features such as quick-info and parameter-help tooltips, completions, go-to-definition, and many more.
Many of us use it every day, but have you ever wondered what’s going on behind the scenes to make it happen?
First stop: types of Intellisense
Intellisense features can be roughly divided into 2 categories:
- Contextual
Can include information from elsewhere, but usually focused on the current position in the text. - Global
Independent of current position in the text (you don’t even need to have an open document).
Contextual Intellisense
- Completions
- Tooltips (signature help / parameter help / quick info)
- Go to definition
- CodeLense
- Editor decorations
- Red squiggle on invalid syntax
Global intellisense
- Diagnostics (i.e. errors and warnings)
- Go to symbol
- Find all references
Next stop: Compilers
Most compilers have 4 stages:
- Lexical analysis (often called lexing)
Processing the source text into a sequence of tokens. - Syntax analysis (often called parsing)
Parsing the tokens to produce a syntactic model, usually an Abstract Syntax Tree (AST). - Semantic analysis
Analysing the syntactic model to determine what things mean.
The semantic model varies from language to language. - Generation (generate appropriate artifacts)
Compilers: a worked example (simplified)
Lexical analysis
The lexer turns a stream of characters into discrete tokens:
Before: _count == 5
After:
Identifier(_count)
Whitespace( )
Operator(==)
Whitespace( )
IntegerLiteral(5)
Syntactic analysis
The parser turns a stream of tokens into an AST.
Before:
Identifier(_count)
Whitespace( )
Operator(==)
Whitespace( )
IntegerLiteral(5)
After:
EqualityExpression
Left
:Symbol(_count)
Right
:IntegerLiteral(5)
Semantic analysis
The compiler refines and interprets the AST.
Before:
EqualityExpression
Left
:Symbol(_count)
Right
:IntegerLiteral(5)
After:
Equality(ExpressionType:Boolean)
Left
:Field(Class1::_count), Type=Symbol(Int32), Target=This
Right
:Int32Literal(5)
Compiler vs. Language Service
- A compiler’s job is to transform the source text and generate outputs.
- A language service’s job is to understand the source text and answer questions about it.
The Compiler’s job
'Foo.cs' -> 'Foo.exe'
'Strings.resx' -> 'Strings.resources'
The Language service’s job
- “What does the identifier at line 3, column 6 mean?”
- “Where is the class
HomeController
used?”
So what does a language service actually DO?
Here’s some source text:
Let’s put on our language-service goggles! What does the language service see?
Tokens
Break up the text into tokens such as:
- LessThanToken (
<
) - NameToken (
Element1
) - GreaterThanToken (
>
) - (etc)
Syntax
Parse the tokens into a syntax tree:
OpenElement(Name=Element1)
OpenElement(Name=Element2)
Attribute(Name=Attribute1,Value=Value1)
EmptyElement(Name=Element3)
EmptyElement(Name=Element4)
CloseElement(Name=Element2)
OpenElement(Name=Element5)
CloseElement(Name=Element5)
CloseElement(Name=Element1)
Semantic Model
Element(Name=Element1,Prefix=,Namespace=None)
Element(Name=Element2,Prefix=,Namespace=)
Attribute(Name=Attribute1,Prefix=,Namespace=,Value=Value1)
Element(Name=Element3,Prefix=,Namespace=)
Element(Name=Element4,Prefix=,Namespace=)
Element(Name=Element5,Prefix=,Namespace=)
How is this useful?
Ask the language service what’s on line 2:
Element(Name=Element2,Prefix=,Namespace=)
Attribute(Name=Attribute1,Prefix=,Namespace=,Value=Value1)
Ask the language service what completions can be offered in the whitespace before Attribute1
:
Containing element is
Element2
, which has 2 possible attributes (Attribute1
andAttribute2
);Attribute1
is already present, so:1 completion:
Attribute2=""
Comments