[git] Implement ability to use the native Git for HTTP(S) connection

Use native command line Git for HTTP connections via GIT_ASKPASS
environment variable, to be able to get rid of the JGit library.

Method description.
The communication logic is very similar to what is already
implemented for SSH connections.
* Define the GIT_ASKPASS environment variable pointing to a script
  generated by the ScriptGenerator.
* When Git requests username and/or password for an HTTP connection,
  it calls the script defined in GIT_ASKPASS.
* The script starts the Java application GitAskPassApp and passes the
  Git request to it as command line arguments. Git process waits for the
  app to return user credentials to the app's stdout.
* GitAskPassApp requests the credentials via XML RPC from the main
  IDEA instance by calling correspondent methods of the
  GitAskPassXmlRpcHandler.
* Before Git command is called, a GitHttpAuthenticator is registered in
  the GitHttpAuthService to receive any XML RPC requests about HTTP
  authentication.
* Once the XML RPC request is received, the GitHttpAuthenticator
  either takes the password from the password safe, or shows a prompt
  to the user in a modal dialog.
* Response is provided back to the GitAskPassXmlRpcClient, which returns
  it to the GitAskPassApp, which in turn returns it to the Git process.

GitHttpAuthenticator details:
* The key for password safe is url + login (see makeKey()).
* If password is asked, then the URL contains the login inside URL,
  and the key is ready.
* If username is asked the key is constructed and is searched for in
  the settings (GitRememberedInputs).
* If no username is specified in the URL, Git queries for the username
  and for the password consecutively. In this case to avoid showing
  dialogs twice, ask for both credentials at once (AuthDialog) and
  remember the password to provide it to the Git process during the next
  request.
* We can't store the credentials enter by user in the settings and
  the password safe right after they have been entered, because user
  can enter incorrect credentials.
  Therefore, listen to the Git process output
  (GitHandler#addAuthListener) and analyze whether authentication
  was successful or failed. If it was successful, ask
  GitHttpAuthenticator to remember the correct credentials, otherwise
  forget them (since they could have been taken from the password safe,
  not from the user).
* Because of this, all other data (url and login, or url with login
  if the password was requested) is saved to the instance fields.

This change fixes all issues opened because of JGit problems:
* authorization issues: IDEA-97178, IDEA-102201, IDEA-100617,
  IDEA-98189
* connection and proxy issues: IDEA-83984, IDEA-92519, IDEA-96205
* IDEA-77411 self-signed server certificates (although we can provide
  more user-friendliness by proposing to set http.sslVerify to false
  in the UI).
* IDEA-100096 client SSL certificates.
* other: IDEA-78744, IDEA-77992, IDEA-76982, IDEA-78310, IDEA-86083,
  IDEA-92163, IDEA-94167, IDEA-99159, IDEA-100097

Known drawback:
No retry. If user enters incorrect credentials,
he will see the failure notification and will need start operation again
to supply correct data this time.
This commit is contained in:
Kirill Likhodedov
2013-03-09 20:49:12 +04:00
parent bc96127b57
commit 296555274e
8 changed files with 581 additions and 0 deletions
@@ -16,6 +16,7 @@
package com.intellij.util.io;
import com.intellij.openapi.util.Pair;
import com.intellij.openapi.util.io.FileUtil;
import gnu.trove.TIntArrayList;
import org.jetbrains.annotations.NonNls;
@@ -147,4 +148,21 @@ public class URLUtil {
public static boolean containsScheme(String url) {
return url.contains("://");
}
/**
* Splits the url into 2 parts: the scheme ("http://" for instance) and the rest of the URL.
* The scheme can be null.
*/
@NotNull
public static Pair<String, String> splitScheme(@NotNull String url) {
final String schemeSeparator = "://";
int ind = url.indexOf(schemeSeparator);
if (ind >= 0) {
String scheme = url.substring(0, ind + schemeSeparator.length());
return Pair.create(scheme, url.substring(ind));
}
else {
return Pair.create(null, url);
}
}
}
@@ -0,0 +1,100 @@
/*
* Copyright 2000-2013 JetBrains s.r.o.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.jetbrains.git4idea.http;
import com.intellij.openapi.util.Pair;
import org.jetbrains.annotations.NotNull;
/**
* <p>This is a program that would be called by Git when an HTTP connection is needed, that requires authorization,
* and if {@code GIT_ASKPASS} variable is set to the script that invokes this program.</p>
* <p>The program is called separately for each authorization aspect.
* I. e. if no username is specified, then it is started and queried for the username, and then started once again for the password.</p>
* <p>Query format is the following:
* <ul>
* <li><code>Username for 'https://bitbucket.org':</code></li>
* <li><code>Password for 'https://bitbucket.org':</code></li>
* <li><code>Password for 'https://username@bitbucket.org':</code></li>
* </ul>
* </p>
* <p>Git expects the reply from the program's standard output.</p>
*
* @author Kirill Likhodedov
*/
public class GitAskPassApp {
// STDOUT is used to provide credentials to Git process; STDERR is used to print error message to the main IDEA command line.
@SuppressWarnings("UseOfSystemOutOrSystemErr")
public static void main(String[] args) {
try {
if (args.length < 1) {
throw new IllegalArgumentException("No arguments specified!");
}
Pair<Boolean, String> arguments = parseArguments(args[0]);
boolean usernameNeeded = arguments.getFirst();
String url = arguments.getSecond();
int handler = Integer.parseInt(getNotNull(GitAskPassXmlRpcHandler.GIT_ASK_PASS_HANDLER_ENV));
int xmlRpcPort = Integer.parseInt(getNotNull(GitAskPassXmlRpcHandler.GIT_ASK_PASS_PORT_ENV));
GitAskPassXmlRpcClient xmlRpcClient = new GitAskPassXmlRpcClient(xmlRpcPort);
if (usernameNeeded) {
String username = xmlRpcClient.askUsername(handler, url);
System.out.println(username);
}
else {
String pass = xmlRpcClient.askPassword(handler, url);
System.out.println(pass);
}
}
catch (Throwable t) {
System.err.println(t.getMessage());
t.printStackTrace(System.err);
}
}
@NotNull
private static String getNotNull(@NotNull String env) {
String handlerValue = System.getenv(env);
if (handlerValue == null) {
throw new IllegalStateException(env + " environment variable is not defined!");
}
return handlerValue;
}
@NotNull
private static Pair<Boolean, String> parseArguments(@NotNull String arg) {
String[] split = arg.split(" ");
if (split.length < 3) {
throw new IllegalArgumentException("Unknown argument format: " + arg);
}
boolean username = split[0].equalsIgnoreCase("username");
String url = split[2];
// un-quote and remove the trailing colon
if (url.startsWith("'")) {
url = url.substring(1);
}
if (url.endsWith(":")) {
url = url.substring(0, url.length() - 1);
}
if (url.endsWith("'")) {
url = url.substring(0, url.length() - 1);
}
return Pair.create(username, url);
}
}
@@ -0,0 +1,80 @@
/*
* Copyright 2000-2013 JetBrains s.r.o.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.jetbrains.git4idea.http;
import org.apache.xmlrpc.XmlRpcClientLite;
import org.apache.xmlrpc.XmlRpcException;
import org.jetbrains.annotations.NotNull;
import java.io.IOException;
import java.net.MalformedURLException;
import java.util.Vector;
/**
* Calls {@link GitAskPassXmlRpcHandler} methods via XML RPC.
*
* @author Kirill Likhodedov
*/
class GitAskPassXmlRpcClient {
@NotNull private final XmlRpcClientLite myClient;
GitAskPassXmlRpcClient(int port) throws MalformedURLException {
myClient = new XmlRpcClientLite("127.0.0.1", port);
}
// Obsolete collection usage because of the XmlRpcClientLite API
@SuppressWarnings({"UseOfObsoleteCollectionType", "unchecked"})
String askUsername(int handler, @NotNull String url) {
Vector parameters = new Vector();
parameters.add(handler);
parameters.add(url);
try {
return (String)myClient.execute(methodName("askUsername"), parameters);
}
catch (XmlRpcException e) {
throw new RuntimeException("Invocation failed " + e.getMessage(), e);
}
catch (IOException e) {
throw new RuntimeException("Invocation failed " + e.getMessage(), e);
}
}
// Obsolete collection usage because of the XmlRpcClientLite API
@SuppressWarnings({"UseOfObsoleteCollectionType", "unchecked"})
String askPassword(int handler, @NotNull String url) {
Vector parameters = new Vector();
parameters.add(handler);
parameters.add(url);
try {
return (String)myClient.execute(methodName("askPassword"), parameters);
}
catch (XmlRpcException e) {
throw new RuntimeException("Invocation failed " + e.getMessage(), e);
}
catch (IOException e) {
throw new RuntimeException("Invocation failed " + e.getMessage(), e);
}
}
@NotNull
private static String methodName(@NotNull String method) {
return GitAskPassXmlRpcHandler.HANDLER_NAME + "." + method;
}
}
@@ -0,0 +1,55 @@
/*
* Copyright 2000-2013 JetBrains s.r.o.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.jetbrains.git4idea.http;
import org.jetbrains.annotations.NotNull;
/**
* This handler is called via XML RPC from {@link GitAskPassApp} when Git requests user credentials.
*
* @author Kirill Likhodedov
*/
public interface GitAskPassXmlRpcHandler {
String GIT_ASK_PASS_ENV = "GIT_ASKPASS";
String GIT_ASK_PASS_HANDLER_ENV = "GIT_ASKPASS_HANDLER";
String GIT_ASK_PASS_PORT_ENV = "GIT_ASKPASS_PORT";
String HANDLER_NAME = GitAskPassXmlRpcHandler.class.getName();
/**
* Get the username from the user to access the given URL.
* @param handler XML RPC handler number.
* @param url URL which Git tries to access.
* @return The Username which should be used for the URL.
*/
// UnusedDeclaration suppressed: the method is used via XML RPC
@SuppressWarnings("UnusedDeclaration")
@NotNull
String askUsername(int handler, @NotNull String url);
/**
* Get the password from the user to access the given URL.
* It is assumed that the username either is specified in the URL (http://username@host.com), or has been asked earlier.
* @param handler XML RPC handler number.
* @param url URL which Git tries to access.
* @return The password which should be used for the URL.
*/
// UnusedDeclaration suppressed: the method is used via XML RPC
@SuppressWarnings("UnusedDeclaration")
@NotNull
String askPassword(int handler, @NotNull String url);
}
+2
View File
@@ -151,6 +151,8 @@
serviceImplementation="git4idea.config.GitVcsApplicationSettings"/>
<applicationService serviceInterface="org.jetbrains.git4idea.ssh.GitXmlRpcSshService"
serviceImplementation="org.jetbrains.git4idea.ssh.GitXmlRpcSshService"/>
<applicationService serviceInterface="git4idea.commands.GitHttpAuthService"
serviceImplementation="git4idea.commands.GitHttpAuthService" />
<applicationService serviceInterface="git4idea.rebase.GitRebaseEditorService"
serviceImplementation="git4idea.rebase.GitRebaseEditorService"/>
<applicationService serviceInterface="git4idea.config.SSHConnectionSettings"
@@ -22,6 +22,7 @@ import com.intellij.openapi.application.ModalityState;
import com.intellij.openapi.components.ServiceManager;
import com.intellij.openapi.diagnostic.Logger;
import com.intellij.openapi.project.Project;
import com.intellij.openapi.util.Key;
import com.intellij.openapi.util.SystemInfo;
import com.intellij.openapi.vcs.FilePath;
import com.intellij.openapi.vcs.ProcessEventListener;
@@ -39,6 +40,7 @@ import git4idea.config.GitVersionSpecialty;
import org.jetbrains.annotations.NonNls;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import org.jetbrains.git4idea.http.GitAskPassXmlRpcHandler;
import org.jetbrains.git4idea.ssh.GitSSHHandler;
import org.jetbrains.git4idea.ssh.GitXmlRpcSshService;
@@ -428,6 +430,18 @@ public abstract class GitHandler {
myEnv.put(GitSSHHandler.SSH_PORT_ENV, Integer.toString(port));
LOG.debug(String.format("handler=%s, port=%s", myHandlerNo, port));
}
else if (myRemoteProtocol == GitRemoteProtocol.HTTP) {
GitHttpAuthService service = ServiceManager.getService(GitHttpAuthService.class);
myEnv.put(GitAskPassXmlRpcHandler.GIT_ASK_PASS_ENV, service.getScriptPath().getPath());
GitHttpAuthenticator httpAuthenticator = new GitHttpAuthenticator(myProject, myState, myCommand);
myHandlerNo = service.registerHandler(httpAuthenticator);
myEnvironmentCleanedUp = false;
myEnv.put(GitAskPassXmlRpcHandler.GIT_ASK_PASS_HANDLER_ENV, Integer.toString(myHandlerNo));
int port = service.getXmlRcpPort();
myEnv.put(GitAskPassXmlRpcHandler.GIT_ASK_PASS_PORT_ENV, Integer.toString(port));
LOG.debug(String.format("handler=%s, port=%s", myHandlerNo, port));
addAuthListener(httpAuthenticator);
}
myCommandLine.setEnvParams(myEnv);
// start process
myProcess = startProcess();
@@ -439,6 +453,33 @@ public abstract class GitHandler {
}
}
private void addAuthListener(@NotNull final GitHttpAuthenticator authenticator) {
// TODO this code should be located in GitLineHandler, and the other remote code should be move there as well
if (this instanceof GitLineHandler) {
((GitLineHandler)this).addLineListener(new GitLineHandlerAdapter() {
private boolean myAuthFailed;
@Override
public void onLineAvailable(String line, Key outputType) {
if (line.toLowerCase().contains("authentication failed")) {
myAuthFailed = true;
}
}
@Override
public void processTerminated(int exitCode) {
if (myAuthFailed) {
authenticator.forgetPassword();
}
else {
authenticator.saveAuthData();
}
}
});
}
}
protected abstract Process startProcess() throws ExecutionException;
/**
@@ -495,6 +536,11 @@ public abstract class GitHandler {
myEnvironmentCleanedUp = true;
ssh.unregisterHandler(myHandlerNo);
}
else if (myRemoteProtocol == GitRemoteProtocol.HTTP) {
GitHttpAuthService service = ServiceManager.getService(GitHttpAuthService.class);
myEnvironmentCleanedUp = true;
service.unregisterHandler(myHandlerNo);
}
}
/**
@@ -0,0 +1,74 @@
/*
* Copyright 2000-2013 JetBrains s.r.o.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package git4idea.commands;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.git4idea.http.GitAskPassApp;
import org.jetbrains.git4idea.http.GitAskPassXmlRpcHandler;
import org.jetbrains.git4idea.ssh.GitXmlRpcHandlerService;
import org.jetbrains.git4idea.util.ScriptGenerator;
/**
* Provides the authentication mechanism for Git HTTP connections.
*/
public class GitHttpAuthService extends GitXmlRpcHandlerService<GitHttpAuthenticator> {
@NotNull
@Override
protected String getScriptTempFilePrefix() {
return "git-askpass-";
}
@NotNull
@Override
protected Class<?> getScriptMainClass() {
return GitAskPassApp.class;
}
@Override
protected void customizeScriptGenerator(@NotNull ScriptGenerator generator) {
}
@NotNull
@Override
protected String getRpcHandlerName() {
return GitAskPassXmlRpcHandler.HANDLER_NAME;
}
@NotNull
@Override
protected Object createRpcRequestHandlerDelegate() {
return new InternalRequestHandlerDelegate();
}
/**
* Internal handler implementation class, it is made public to be accessible via XML RPC.
*/
public class InternalRequestHandlerDelegate implements GitAskPassXmlRpcHandler {
@NotNull
@Override
public String askUsername(int handler, @NotNull String url) {
return getHandler(handler).askUsername(url);
}
@NotNull
@Override
public String askPassword(int handler, @NotNull String url) {
return getHandler(handler).askPassword(url);
}
}
}
@@ -0,0 +1,206 @@
/*
* Copyright 2000-2013 JetBrains s.r.o.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package git4idea.commands;
import com.intellij.ide.passwordSafe.PasswordSafe;
import com.intellij.ide.passwordSafe.PasswordSafeException;
import com.intellij.ide.passwordSafe.impl.PasswordSafeImpl;
import com.intellij.ide.passwordSafe.ui.PasswordSafePromptDialog;
import com.intellij.openapi.application.ApplicationManager;
import com.intellij.openapi.application.ModalityState;
import com.intellij.openapi.diagnostic.Logger;
import com.intellij.openapi.project.Project;
import com.intellij.openapi.util.Pair;
import com.intellij.openapi.util.text.StringUtil;
import com.intellij.util.AuthData;
import com.intellij.util.io.URLUtil;
import com.intellij.vcsUtil.AuthDialog;
import git4idea.jgit.GitHttpAuthDataProvider;
import git4idea.remote.GitRememberedInputs;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
/**
* <p>Handles "ask username" and "ask password" requests from Git:
* shows authentication dialog in the GUI, waits for user input and returns the credentials supplied by the user.</p>
* <p>If user cancels the dialog, empty string is returned.</p>
* <p>If no username is specified in the URL, Git queries for the username and for the password consecutively.
* In this case to avoid showing dialogs twice, the component asks for both credentials at once,
* and remembers the password to provide it to the Git process during the next request without requiring user interaction.</p>
* <p>New instance of the GitAskPassGuiHandler should be created for each session, i. e. for each remote operation call.</p>
*
* @author Kirill Likhodedov
*/
class GitHttpAuthenticator {
private static final Logger LOG = Logger.getInstance(GitHttpAuthenticator.class);
private static final Class<GitHttpAuthenticator> PASS_REQUESTER = GitHttpAuthenticator.class;
@NotNull private final Project myProject;
@Nullable private final ModalityState myModalityState;
@NotNull private final String myTitle;
@Nullable private String myPassword;
@Nullable private String myPasswordKey;
@Nullable private String myUrl;
@Nullable private String myLogin;
private boolean myRememberOnDisk;
GitHttpAuthenticator(@NotNull Project project, @Nullable ModalityState modalityState, @NotNull GitCommand command) {
myProject = project;
myModalityState = modalityState;
myTitle = "Git " + StringUtil.capitalize(command.name());
}
@NotNull
String askPassword(@NotNull String url) {
if (myPassword != null) { // already asked in askUsername
return myPassword;
}
String prompt = "Enter the password for " + url;
String key = adjustHttpUrl(url);
myPasswordKey = key;
return PasswordSafePromptDialog.askPassword(myProject, myModalityState, myTitle, prompt, PASS_REQUESTER, key, false, null);
}
@NotNull
String askUsername(@NotNull String url) {
String key = adjustHttpUrl(url);
AuthData authData = getSavedAuthData(myProject, key);
String login = null;
String password = null;
if (authData != null) {
login = authData.getLogin();
password = authData.getPassword();
}
if (login != null && password != null) {
myPassword = password;
return login;
}
final AuthDialog dialog = new AuthDialog(myProject, myTitle, "Enter credentials for " + url, login, null, true);
ApplicationManager.getApplication().invokeAndWait(new Runnable() {
@Override
public void run() {
dialog.show();
}
}, myModalityState == null ? ModalityState.defaultModalityState() : myModalityState);
if (!dialog.isOK()) {
return "";
}
// remember values to store in the database afterwards, if authentication succeeds
myPassword = dialog.getPassword();
myLogin = dialog.getUsername();
myUrl = key;
myRememberOnDisk = dialog.isRememberPassword();
myPasswordKey = makeKey(myUrl, myLogin);
return myLogin;
}
void saveAuthData() {
// save login and url
if (myUrl != null && myLogin != null) {
GitRememberedInputs.getInstance().addUrl(myUrl, myLogin);
}
// save password
if (myPasswordKey != null && myPassword != null) {
PasswordSafeImpl passwordSafe = (PasswordSafeImpl)PasswordSafe.getInstance();
try {
passwordSafe.getMemoryProvider().storePassword(myProject, PASS_REQUESTER, myPasswordKey, myPassword);
if (myRememberOnDisk) {
passwordSafe.getMasterKeyProvider().storePassword(myProject, PASS_REQUESTER, myPasswordKey, myPassword);
}
}
catch (PasswordSafeException e) {
LOG.error("Couldn't remember password for " + myPasswordKey, e);
}
}
}
void forgetPassword() {
if (myPasswordKey != null) {
try {
PasswordSafe.getInstance().removePassword(myProject, PASS_REQUESTER, myPasswordKey);
}
catch (PasswordSafeException e) {
LOG.info("Couldn't forget the password for " + myPasswordKey);
}
}
}
/**
* If the url scheme is HTTPS, store it as HTTP in the database, not to make user enter and remember same credentials twice.
*/
@NotNull
private static String adjustHttpUrl(@NotNull String url) {
String prefix = "https";
if (url.startsWith(prefix)) {
return "http" + url.substring(prefix.length());
}
return url;
}
@Nullable
private static AuthData getSavedAuthData(@NotNull Project project, @NotNull String url) {
String userName = GitRememberedInputs.getInstance().getUserNameForUrl(url);
if (userName == null) {
return trySavedAuthDataFromProviders(url);
}
String key = makeKey(url, userName);
final PasswordSafe passwordSafe = PasswordSafe.getInstance();
try {
String password = passwordSafe.getPassword(project, PASS_REQUESTER, key);
if (password != null) {
return new AuthData(userName, password);
}
return null;
}
catch (PasswordSafeException e) {
LOG.info("Couldn't get the password for key [" + key + "]", e);
return null;
}
}
@Nullable
private static AuthData trySavedAuthDataFromProviders(@NotNull String url) {
GitHttpAuthDataProvider[] extensions = GitHttpAuthDataProvider.EP_NAME.getExtensions();
for (GitHttpAuthDataProvider provider : extensions) {
AuthData authData = provider.getAuthData(url);
if (authData != null) {
return authData;
}
}
return null;
}
/**
* Makes the password database key for the URL: inserts the login after the scheme: http://login@url.
*/
@NotNull
private static String makeKey(@NotNull String url, @NotNull String login) {
Pair<String,String> pair = URLUtil.splitScheme(url);
String scheme = pair.getFirst();
if (scheme != null) {
return scheme + login + "@" + pair.getSecond();
}
return login + "@" + url;
}
}